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DEVELOPMENT OF GUIDANCE MEASURES FOR YOUTH- 



WORK TRAINING PROGRAM ENROLLEES 
PHASE I: r^ASUREMENT OF PROGRAM OBJECTIVES 

AND THE DEVELOPMENT OF CRITERIA 
Norman E. Freeberg 
and 

Richard R. Reilly 
Educational Testing Service 
Princeton 5 N. J. 

Abstract 

This study deals with methodological issues in the development of 
measured objectives for a youth --work training program, along with analyses 
that serve to illustrate hov empirical data can be used to supplement 
rational choices of program outcomes. Correlational analyses of outcome 
measures -were used to define the "structure” underlying a system of criteria 
from which the best sets of coherent scales, or composite measures, might 
be derived . 

For end*-of -training program (short-term) objectives, the dominant 
dimension found was defined as "Training Program Adjustment" consisting of 
ratings of enrollee performance in the program, combined with measures of 
social adjustment. Other distinct groupings of variables were definable 
as "Job Aspirations," "Social Adjustment," "Personal Image" and "Planning 
Competency . " 

Longer term (intermediate) objectives applicable to enrollee post- 
program performance — for those who had held full-time employment — produced 
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rela.tively clear patterns of ,jot>-*oriented capabilities. Among these, a 
"General Job Success and Adjustment" pattern vas dominant , with others 
definable as "Job Stability-Mobility," "Striving for Vocational Success" 
and "Blue Collar Job Success." Two personal adjxistment dimensions 
accompanying those were designated as "Coimnur'.ity and Family Adjustment" 
and "Vocational Adjustment— Dissatisfaction . " 

A set of four dimensions, obtained from the criterion variables 
applicable to a subgroup of former enrollees who had never held f\jll-time 
employment, was found to be relatively unclear and difficult to define. 
The research implications and problems posed by attempts to measure degree 
of "success" for such a subgroup are considered. 

Some descriptive highlights concerning vocational behaviors of former 
enrollees are presented and future research needs , essential to a more 
complete under st anding of program objectivess are discussed. 
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DEVELOH^ENT OF GUIDANCE MEASURES FOR YOUTH- 



WORK TRAINING PROGRAM ENROLLEES 
PHASE I: MEASUREMENT OF PROGRAM OBJECTIVES 

AND THE DEVELOPMENT OF CRITERIA 

I . luLroduct ion 



A. Pux-pose 

The ability to demonstrate the effectiveness of any social program 
is almost completely a function of the ability to measure intended goals , 
or objectives 5 with reasonable a,ccuracy. It is the measurement "quality” 
of the variables chosen to define ■ performance standards that *nltimately 
makes it possible to: (a) specify the degree of program success, (b) 

feedback information for modifying and improving program components and 
(c) carry out meaningful research leading to increased understanding of 
the population served. Where the effort is made to identify and measure, 
properly 5 an array of agreed-upon outcome valuables there are corre- 
sponding administrative advantages likely to accrue in the form of better 
control of prograiu processes , the design of more effective programs and 
the establishing of accountability for those programs that have been 
undertaken . 

For. many governmentally-funded manpower training programs, aimed at 
effecting broad social change, statements of program intent have often 
been highly'' general and too rarely translated into applicable behavioral 
objectives — ^both long and short term — with defined measurement requirements . 
Even less attention has been paid to the availability and suitability of 
outcome variables that would be needed to measure intended goals with 
reasonable continuity from study-to-study , or program to program. Early 
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effortis "bo demon s*b raise israining ouis comes were primarily anecdoisal (U,S. 

Senaise 5 1963) and heavily dependent on testimonials that can hardly serve 
as strong supporting evidence for clear specifications of goals - Relatively 
recent studies, conducted in the latter half of the 1960 * s , have tended 
to move toward more precisely defined outcomes (Levitan & Mangum, 1969)- 
Unfortunately, they have neglected to provide a hroad overview of possible 
measures for satisfying a variety of criterion needs or a stated rationale — 
logical or empirical — *to Justify the choice of specific variables - In 
addition, the studies have generally not gone beyond descriptive surveys 
of demographic data (e.g., numbers working or looking for work by age , 
sex, educational level, etc.) with simple tabulations of fi'eq^uencies , or 
proportions 5 for the criterion variables chosen, so that rather weak 
evidence is available for selecting one particular set of outcpme vari- 
ables as more appropriate than others. Borus and Tash (l9T0) in their dis- 
cussion of impact measurement for manpower training programs have leveled 
similar criticism at these study designs and the virtually exclusive use of 
univariate methods. Fortunately, there are more fruitful analytical approaches 
available in behavioral science that would seem to be essential if the hope 
is to construct a defensible system of measurable program objectives , able 
to encompass the complex performance variables that characterize the manpower 
training enterprise. 

The present study represents an initial step in demonstrating how certain 
of those techniques might be applied to a systematic, comprehensive exam- 
ination of a number of criterion measures for that class of manpower train- 
ing programs that have come to be designated as "youth work training programs" 
(Herman & Sadof sky , I966) . The majority of such programs, which have been 
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under federal sponsorship, are intended to serve yoimg adults from poverty- 

level hackgroimds who are considered socially and culturally disadvantaged. 

Although at least one large-scale program has had as its primary short-term 

objective that of preventing yoimgsters from dropping out of school (.i.e., 

the Wei^horhood Youth Corps , In School Program) , most of the programs 

are concerned with enhancing the "employability of those who have left 

* 

the formal academic setting. 

Through an analysis of various specific outcomes — Abased on data 
obtained from one youth-work training program considered reasonably repre- 
sentative in its design and enrollee population (i.e.. Neighborhood Youth 
Corps, Out of School Program) — it is expected that a set of guidelines 
for choosing the most useful measures of program "success" can be formulated 
and applied: (a) in evaluating the effectiveness of this, or similar, 

youth-work training programs, (b) in the validation of tests or other 
predictive devices for selection, guidance, or placement (e.g., paper and 
pencil tests, ratings, work samples, etc.) and (c) when relevant performance 
measures (i.e., dependent variables) are to be selected in the planning 
and conduct of research programs. 

Within the primary study purpose , it should be clear that it is impract 
cal for any single criterion study to pin down all outcome variables perti- 
nent to virtually all youth training programs (present or prospective). 

Such programs not only possess a number of variations in their short-term 
intentions and training methods, but are modified frequently enough, in 



"^For descriptions' of the various youth-work training programs and their 
stated, or implied, goals see Mangum ( 1969 ) and Levitan and Mangum ( 1969 ). 





-U- 



•bhose respectis , "bo ■warran'b a degree oT con’binual upda*bing for "bheir 
sysfems of outcome measures. Whaf does exisf , however, is fhe opporfunify 
*bo demonsfrafe one model for prograramafic criterion research, based on 
variables with the broadest applicability across programs (i.e., *'core*' 
criteria) and to help clarify crucial issues in setting standards that 
must concern any youth-work training program forced to deal with c .mplex, 
behaviorally-based , -''1: ,lves. 

B . Background 

The necessity for u^_dertaking the development of suitable criterion 
measures for youth-work training programs arose from the more iirmiedxate . 
requirement to validate a battery of measures intended for use as guidance 
and counseling tools » The instriiments that comprise the battery deal with 
various attitudinal, vocational and reasoning skills, and were designed 
explicitly for use with disadvantaged adolescents of relatively low verbal 
skill levels enrolled in the Neighborhood Youth Corps program (preeberg, 

1968 ; Preeberg, 19T0)- Tn a preliminary development phase, various test 
characteristics were examined with external validity assessed solely on 
the basis of available, concurrent criteria in the form of coimselor and 
work supervisor ratings . 

Validation using one type of short-term criterion is inherently limiting. 
It is far more desirable in any test development effort for the value of the 
appraisal tools to rest on a variety of as long range and relevant measures 
of prograxn outcomes as possible. For example, the choice of counselor 
and work supervisor ratings should be predicated on their demonstrated value 
as outcome measures which should, in turn, be determined by their relation 
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( or relevance) to other longer-term outcome measirres that may consist 
of a variety of vocational and social adjustment variables - 

When fac 1 with the task of choosing criteria for a youth-work training 
program, however, it became apparc r^t there was little coherent research 

material available to provide suffi lent inowli :dge of the "criterion 
domain" from wxiich one could select efer. sible variables of known me asure- 
ment characteristics and accessibilit r , ,iithc suitable information to 
define the value of specific criterion i asure . research or evaluation 
studies involving work -training program .rollaas have been forced to depend 
almost entirely on rational bases (usa: _y nca made explicit by the 
investigator) for the choice of what can be considered \intested dependent 
variables. Thus, in some studies, scaled values of "self-esteem", "work 
motivation" or "social adjustment" have been embodied in one, or a few, 
questionnaire items. In others, more formally developed scales may be 
applied or the investigator may choose to infer the same constructs exclu- 
sively on the basis of observation by others (e.g., employers, family 
members, peers)j^from records of police contacts, number and types of Jobs 
held, etc. Any, or all, such variables may be of value. The point is that 
there has been no systematic,' empirical test of that value, in terms of 
measurement properties and relationships between criterion variables , 
essential for determining the dimensions being tapped by any set of outcome 
measures. It is this lack of available evidence to support the investi- 
gator’s choice of particular criteria, from the pool of potentially usable 
outcome variables, that can often lead to ambiguous measurement with consequent 
weakening of comparability, or con~ir\ilty, in the criteria chosen end in 
the conclusions reached by different s“udies. 
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The efifec'b is perhaps more acuhely fel*b in a't'bemp'bs ho conahruch and 
validate predictive appraisal devices or tests , because the suitability 
of the criterion measures (i.e., their availability, -^liability , relevance, 
bias, factor composition) can be reflected directly • d immediately in 
the choice of test content and the levels of validz-ty chievable (Thorndike, 

I9U9) • 

Since any youth-work training program must depend on behavioral assess- 
ment (i-e., tests and measures) fcir conducting its guidance and research., 
or for evaluating its impact , the conclusion is inescapable that any 
measures used for such a program ^^can be no better than the criteria which 
define it^' (Krug, I 961 ) - This central role for criteria in the development 
and application of any measurement system has been stressed repeatedly in 
the psychometric literature, although often slighted in practice to the 
detriment of various social and educational programs . It is under st andably 
difficult and costly to accept Guilford ^s (195^) recommendation that ^^as 
much time be spent on developing criteria to validate tests as on the tests 
themselves” since, as he admits, this means ”doing the hard things first” 
(Guilford, 1954, Ch . l4 ) . Nevertheless, repeated failure to deal explicitly 
with criterion problems has left major efforts in behavioral appraisal 
and evaluation with a legacy of \Hirealized improvements in validity: 
improvements that are becoming more necessary to achieve if widely used 
appraisal tools are ever to meet long standing vocational needs and if the 
legal r eq,uirement s for demonstrated validity of tests are to be fulfilled 
( Polermo , 1969 ) • 
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By a'btending to criterion variables at this stage in the development 
of* guidance and coxmseling tools » it is hoped that, in addition to demon- 
strating the advantages of proper choice and measurement of* program goals . 
costly criterion inadequacies vhich might subsequently affect large- 
scale test validation efforts can be minimized. 

^ ' Concepts of Criterion Development 

In order to provide appropriate perspective for the purposes and 
techniques of the present study, it is of value to review customary 
approaches to criterion development along with some of the complex issues 
that can arise when these are applied to .the definition and measurement 
of work-training program goals. Although considered separately below, 
these issues and concepts are highly interrelated and should be viewed . 
as different facets of an overall process of criterion determination. 

1. Single vs. Multiple Criteria : The search for a single criterion 

as the standard of "success for any educational or social program, 
appears deceptively appealing, but can prove difficult to justify in its 
application. Typically, for youth— work training programs, the single 
outcome of choice used to represent an "employability" objective has been 
a frequency count of the number of enrollees .who obtain employment over 
some defined period following program completion. Such a measure has 
rati on a3- justification and certainly represents one of the more important 
program outcomes . But , whether that or any other single criterion va-riable 
can serve as an ultimeite standard, optimally informative of program 
accomplishment, is questionable for several reasons. First, it is 
generally untrue that a single measure such as a job -count index, by itself . 
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possesses the all-encompassing criterion value assumed. II, or exa.iple , 
most ex-enrollees of youth-work training programs had ottain- employment 
at such jobs as laborers or dishwashers, a simple employment mant alone 
would be considered by most observers to represent an Inc omp. ~ ^e , if not 
misleading, standard of program accomplishment. Additional c ocome lata 
that are job-related would obviously be called for in the for a of tne 
"quality" of the jobs obtained, and based perhaps on job leve__, salary, 
advancement possibilities, length of employment, etc. In practice, there- 
fore, even those who deal with, or espouse, a single goal for a work- 
training program often Intend to consider at least seven'll specific 
variables that constitute a broader employment dimension. 

A second reason why it is questionable to choose a single prograii goal 
is that it is self-defeating to put all of the "criterion eggs" in one 
basket, especially when that single objective Involves a vocational measure. 
Types of jobs and their availability can vary regionally, rieasonally , on 
the basis of changing local or national economic conditions and in the 
patterns of minority-group discrimination Imposed. A training program that 
had properly inculcated in its enrollees the means and the desire to obtain 
and hold a job might, therefore, be judged unfairly where that judgment 
hinged on one specific criterion measure (or class of criteria) affected 
by a number of conditions beyond the control of the program under evalua- 
tion. In order to overcome such biasing effects, it seems logical to 
consider additional Important objectives, other than those that deal with 
purely vocational, "job-getting" outcomes. This can only be done by 
demonstrating changes in the characteristics of the individual enrollee 
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(e.g.5 in hxij^ work motivation ^ vocational planning atllityj personal 
adjustment 5 acauisition of intellectual or technical skills, etc • ) 5 
all of which might have heen favorably influenced hy the program — and 
highly related to later success in obtaining Jobs — but not verifiable 
where there is total dependence on a single Job-count index at some 
narrowly defined point (or period) in time. 

A third shortcoming ,in being dependent primarily on a single criterion, 
is that the one form of measured outcome becomes the basis fox'* the program’s 
existence™i . e . , the criterion defines the program . Thus, whatever its 
title might imply, a training program that seeks to measure as its sole 
outcome scores on reading achievement level or, perhaps, the number of 
enrollees who remain in school can, by definition, be considered only 
a "remedial reading" program or a "school-retention" program — nothing morel 
Assumptions about other outcomes that mi^t enter into, or be associated 
with, the single criterion chosen are purely gratuitous if they are unmea- 
sirred and undemonstrated. In this context, it should be apparent that a 
relatively simple, unitary criterion is also "unrealistic. People who design 
a youth work training program and those responsible for its conduct (e.g., 
counselors and work supervisors) do not behave in practice, as if there 
is one single overriding outcome to be achieved. They seek instead to 
effect many different enrollee behavioral changes of varied, often undeter- 
mined , value . 

A fourth, and probably a major drawback in resorting to a single 
criterion, stems fi'om the restrictions imposed in understanding how a 
training program is functioning or succeeding. That is, it limits avail- 
able knowledge regarding which particular program components mi^t be 
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differentially affecting which aspects of the criterion. Dimnette (19 d 3) 
emphasizes essentially the same point in discussing the design and appli- 
cation of measures for predicting Joh success, when he argu.es that only 
hy the use of complex, multiple criteria can one "learn more about the total 
constructs" that are (or should he) measured hy tests. It seems equally 
likely that the functional complexities of a youth-work training program 
would best lend themselves to appraisal by more than a relatively simple, 
single criterion score. 

2 . Criterion Measures over Time (Proximal, Distal and Ultim ate Goals. 

The choice of any goals and their measurement can only have meaning 

along a time continuum.. Certain goal formulations imply a relatively 

immediate capability to measure the outcomes specified; others imply 

relatively lengthy time periods before information bearing on the outcomes 

can be made available. Major issues of criterion definition, cost, accessi- 

Tailify and demonstrated value are influenced by the time constraints imposed 

and are conventionally considered under temporal designations of the criteria 

as "proximal", "distal" or "ultimate" measures. 

a. ProxiTnal or short-term criteria are those considered most 

immediately available and usually the most accessible or 

easily obtained. In a youth— work training program, such 

as Keighborhood Youth Corps , these mi^t be available at 

* 

the time the enrollee is "completing" the program and 



Program "completion" is a relatively arbitrary term since the enrollees 
can, and do, leave the program as they choose. The nominal enrol.lment 
period designated for the Neighborhood Youth Corps out-of-school program, 
lio'wever 5 is one of* six mon'C.Jis . 
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consist of q.iaant itative ratings ty various px ugx am 
perscnnel (e.g-, =ow>s=Xors and vork suparvis-ors ), peers (e.g. 
fellow enrollees), self-evaluations (e.g., attltudinal 
measures) or forms of administrative information re- 
corded ty the project (e.g., work site absences, number 
of work site assignments). 

The distal or intermediate term criteria would be those 
measures available within some reasonable period after 
the enrollee has left the program and might include 
various aspects of employment, vocational, social and 
family adjustment, motivation or attitudes and infor- 
mation on specific post-program experiences such as 
contacts with legal authorities-, use of USTES, etc. 

It is, certainly, the ultima te or long-t e^ criteria 
that represent the most desirable ones conceptually but 
the most difficult about which to obtain information 
(often beyond the bounds of practical capability) . These 
tend to be the criteria for which the broad dimensions 
can often be agreed upon— e.g., "vocational success" or 
social "adjustment"-but the most difficult to define 
explicitly. Just what constitutes vocational success 
over many years or a working lifetime is not an easy 
task to determine. Collection of such ultimate forms of 
criterion information may require essentially prohibitive 
expenditures of effort and money be, ad what is considered 
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justif iable for program purposes. Or, similarly, it 
- may te considered too amtitious , if not logically 
inappropriate, to ass-ume that the effects of any youth 
■work training program could he expected to extend heyond 
a period of several months , or a fe"w years , at most. 

In that case, goal statements should he translated into 
measures that are **less ultimate'* and set "within a more 
appropriate, arbitrarily agreed-upon time span. The 
shorter that time span the more obtainable some foims 
of long range criteria are likely to be. 

By dealing "with criteria in this temporal framework, there are improved 
possibilities for \inderstanding the measures chosen through recognition of 
their overlap and their interrelationships . In addition, it becomes clearer 
in formulating and applying criteria — especially the immediate and inter- 
mediate ones — what assumptions are being made, rationally and empirically, 
regarding the "respr esentativeness ** of each criterion measure for some form 
of longer range goal intention. The practical advantage gained is expressed 
most clearly in the statement that': **As one moves from the ultimate towards 

more and more immediate criteria, there will be more and more room for 
statistical considerations to supplement the rational in evaluation of the 
proposed criterion measures** (Thorndike, 19^9 5 P* 123). It is only through 
such statistical considerations that criterion representativeness and value 
can eventually be determined. 



V ' 
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3. Measurement Concepts : For criteria, as for any test ins tr lament s , 

the hasis for selecting the most functional variables should stem from 
knowledge of their measurement characteristics. They should, in other 
words 5 he reliable or reproducable , relevant to other criteria (i.e., 
"valid" in the conventional sense) and as objective, or bias-free, as 
possible . 

a. Relev anc e or e qui valenc e : Given any defined outcome 

measures at any proximal point in time there remains an 
obligation to demonstrate the relevance of those measures 
to longer-range criteria. Practical needs to substitute 
the more quickly and cheaply obtainable criteria for the 
costlier 3 longer-term, varieties can only be met by 
reasonable evidence to support those choices intelligently. 
This, in effect, constitutes what Wherry (195T) has stressed 
as a need for "intelligent substitution" of the more obtain- 
able (short-term) criteria for the more distal ones. In 
the case of youth-work training program, various short- 
term criteria derived from ratings by others , enrollee 
selfr appraisals and attitudinal scales , measures of various 
skills acquired, etc., may be decided upon as relatively 
easy to obtain and reasonably Justifiable at the time the 
enrollee has completed, or is leaving, the program. 

However, the most important problem remaining to be solved 
would be one of specifying the extent to which each 
criterion measure at program completion is equivalent 








(relevant) to outcomes deemed more important at later 
points in time (i.e., longer--term patterns of vocational 
success and social adjustment). In order to accomplish 
this 5 it is possible to compare underlying dimensions 
for the same or similar criteria obtained, at different 
points in time, for different individuals in a cross- 
sectional comparison. Far more definitive and useful 
(although costly) -would be use of a follo-w-ups 
longitudinal sample that allowed for direct correlations 
between earlier and later scores on criterion measures 
obtained for the same individuals . ¥hen immediate criteria 
are found virtually unrelated to longer-term meas-ures 
is very difficult to justify their use on any basis, 
however easily obtainable they may be and whatever the 
rationale for their choice or the degree of objectivity 
they may be shown to possess. 

b. Reliability : Although not as critical to criterion determina- 

tion as relevance, the reliability of any criterion measure 
serves as a limiting value which can attenuate its relationship 
to other, more reliable, criteria (Thorndike, 19^9 » Ch. 5). 

Some moderate level of consistency should be demonstrated 
if the measure is to have any value in support of some 
broader criterion construct. The ability to deraons-trate 
performance consistency by measuring and remeasuring the 
same behaviors over a short time period (i.e. , analogous to 
test— retest reliability) rex>resents a lux-ury that is usually 
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imprac'bical to attain I'^or many of the criteria that 
would he applicable to a youth work training program 
(e,g-5 length of s tay on first joh). More often, 
reliability of the criterion measure can only be 
demonstrated by shoving the internal consistency of a 
multi-item scale and by the relationships between various 
criteria^ grouped either on an ^ priori logical basis (e.g., 
"motivation” as reflected in nimaber of Jobs the individual 
tried for, visits made to employment agencies, responses 
to appropriate questionnaire items), or from factors on 
which similar criterion measures are shown to cluster. 

Therefore , the wider the variety of desirable long-range 
criteria that can be applied, the greater the opportionity 
to increase reliability of a given criterion construct by 
increasing the range of behaviors sampled, 
c. Bias : Adverse effects on both the relevance and reliability 

of a criterion can stem from systematic biases that 
differentially influence scores obtained from various sub- 
groups of the population being considered- For criterion 
data obtained on the performance of present or former youth- 
work training program e nrollees , subgroup biases may be 
introduced in various measirres of vocational success because 
of different employment opportunities in various geographic 
areas , differences in professional capability in rater 
leniency, or in training opportunities from project to project. 
Many significant biases can be difficult to identify, requiring 
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extensive research efforts to do so, while others 
may he difficult to correct or impractical to eliminate 
even when identified. Where a particular form of 
identified hias cannot he eliminated, its effects can 
he minim_lzed hy randomizing its occurrence through 
all subgroups to as great an extent as possible, or 
by removing its effects statistically. A range of 
possible biasing effects to be aware of in the develop- 
ment of criteria have been outlined and discussed by 
Br Ogden and Taylor (1950), under categories of "deficiencies" 
in choosing pertinent criterion elements, "contamination" 
by the introduction of extraneous elements, "inequality" 
of scale units and "distortion" resulting from improper 
weighting of criterion elements , 

Criterion Availability and Practical Constraints : Despite an 

awareness of proper approaches to definition ,and measurement of criteria, 
there are additional issues that can impose overriding, practical limita- 
tions on the most carefully designed attempts to develop or apply those 
measures. Among these are: 

a. The accessibility of certain forms of desirable cri- 
terion data and associated costs required to obtain the 
information may be prohibitive enough to make use of the 
measures impractical and require the search for sub- 
stitute measxires. Longer-term criteria for a youth- 
work training program are the ones that can require 
pai'ticular time and effort in their collection, since the 
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individuals about whom post -program performance data 
are to be collected must be located (often at very high 
cost) and, once located, be willing to provide the 
required information. Where it is impractical to obtain 
information directly from an. enrollee or e^r-enrollee , and 
where corroborative information is sought, it may be 
necessary to utilize information from pub"' ic sources 
(e.g., law enforcement agenc ::S . local state and fed- 
eral agencies , credit agenciez: . or from employers and 
other obser"'ers (e.g., friea family meri-oers ) . Even 
for those "external'' sources, ..^cwever , accessibility 
can remain a problem in term jf cost, accuracy and, 
e qually i mpor t ant, in t e rms o i' eth i c al c on s i de r at i on s . 

Tiler e may, for examplei> be some reluctance on the part 
of an interviewer (as well as other persons, and agencies) 
to violate the individual’s ri^t to privacy by 
providing information with, or without, his knowledge — 
and rightly so- The ramifications of these point Ss which 
are only mentioned in passing here, ai;*e critical to all 
aspects of research and evaluation with youth -work 
training programs and will be considered in detail in 
Section II of this paper. 

b. The ob^j ectivity or sub,-] ectivity of criterion measures are 
a function of the nature of the program and the degree of 
information accessibility. Ideally, the more objectively 
verifiable the outcome measures chosen the more accurate 
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they are assumed to he; while the greater the human 
intervention involved the greater the subjectivity 
and the greater the assumed chances Tor inaccuracies - 
Those criteria hipest on objectivity would be the ones 
based on routinely and easily recorded perf*orma2:.ce 
indices (e.g., absences, units of* production ouxput on 
a Job). Further away from highest level of* objectivity 
can be found the systemsLtically derived observational 
data dealing with reasonably observable aspects of* per- 
formance (e.g., routine supervisor ratings of* Job 
competence). Still less objectivity may be assigned to 
factual statements, by the individual, about himself — 
but potentially capable of being checked for accuracy 

police contacts, salary); with the more subjective 
portion of the continuui^i reserved for self- evaluation in 

the form of attitudinal measures (e.g., vocational plans, 

■K- 

motivation, family adjxistment , Job satisfaction, etc.). 

The more subjective the criterion sources the greater 
the care necessary to guard aga-inst measurement inaccuracies 
and to demonstrate their relevance to (i.e., correlations 
with) the more objective outcomes • 



*Purely anecdotal, unstructured, response information (whether observations 
or self— evaluative ) would represeni; the extreme- subjective end of an objective- 
subjective continuum, hut is not considered pertinent to the concepts of cri- 

development presented here as long as the information remains nonsystems-i 
in its collection and unquairbifled in its application. 
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c. The. g-eneraliz; ability oT any criteria constitutes an 
important constraint in their applicability to a 
broad class of programs or evaluative needs. Where 
broad goals chitting acr"oss similar programs can be 
igreed upon and the samples involved remain reason- 
.ibly comparable, some form of "evaluative continuity’’ 
in the use of generali zable outcomes can be an 
:Lchievement of obvious importance. Part of any 
criterion package she rid certainly include a variety 
of specific criteria where they are logical for differ- 
ential program intentions. That is, some measure(s) 
of verbal skill should be applied vhere remedial train- 
ing constitutes a program component; or, for measuring 
specific areas of technical skill vhere those are being 
taught. But, the primary risk in too great dependence 
on criteria that are highly program— specif ic‘ for a 
conceptually similar group of programs (e.g., manpover 
training) is that vhere program components are contin— 
\ially changing^ through practical necessity or other 
e:icternal demands (e.g., job markets demanding entirely 
nev technical skills), the evaluation system assumes 
the costly risk of continually "chasing" those changing 
program goals. 

Despite the practical desirability of using videly 
applicable criteria, the extent to vhich that intention 
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can "be accomplished is often further limited hy what 
lias heen termed the dyn ami c q.ual i b y of measured c zt- 
comes (Bass, 19^2; Ghiselli , I960;, This goes beyc.: I 
intentional changes in program desig ' and associate . 
criteria dictated by feedback of res Hts • The cone 
has been used to refer to changes over time in 
the measurement characteristics of the same, or similar, 
criteria. For example, a battery of guidance tests may 
be reasonably predictive of work supervisors ' rating;^ 
at the completion of the training program, drop to : 
much lower level of validity for on-the-job performance 
ratings six months after the enrollee leaves the program 
and then show a return to the initially higher levels 
perhaps a year and a half later (using essentially the 
same rating criterion). Thus, in choosing outcome 
measures there maybe a built-in constraint .to narrow 
those choices to criteria that are most stable in terms 
of predictability and relevance at different points over 
time — or at least to know which criteria are subject to 
such dynamic change and what the criterion ”mix” should 
be at each segment in time. 

e. Existing program demands constitute the last of the constraints 
considered here and can be viewed as primary and pervasive 
in the choice of criteria for study. The constraint stems 
from the necessary, logical, connection between objectives. 
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program content and.;,cperat ion . Although it has heen 
pointed out that the criteria chosen can he said ”to 
define the program/^ it is eq.ually true that, once a • 

youth--worh training program is operational , criterion 
choice is limited for practical purposes hy the nature 
and conduct of that program. The criterion measiJires 
must, under such a circumstance, he selected -with the 
intention of reflecting program components to a reason- 
able extent, since there is little logic in measuring 
outcomes not likely to he (nor claimed to he) affected 
as a result of participation in the program. As a somewhat 
extreme example , one mi^t obtain evidence to show improve- 
ments in dental hygiene resulting from enrollment , in a 
non resident ial,youth-work training program hy measuring 
explicit v£iriahles reflecting the enrollee’s dental status - 
Although a desirable outcome, its logical choice as a 
criterion of priority could he challenged if there is no 
information concerning dental habits provided as a program component; 
nor any attempt . made to provide dental care directly . 

All of the above concepts in criterion development should eventually 
enter into the design of a total and flexible system of outcome measures. 

Where the data of the present study are available to do so , these concepts 
will be applied in guiding this initial overview of criteria for use with 
a youth-’Work training program- 
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II . Method 

k , • election of Criterion Variables 

In order to measure objectives for a youth -work training program 

l. -’ : necessary to design an initial set of potentially useful criterion 

'.lesj from which the best ones can be pruned as a result of empirical 

*;:es. The intent is to cover an adeq^uate range of variables 

b .. on a suitable rationale t a step that has been stressed 

ucial 5 if misleading criterion dimensions are to be avoided (Nagle, 

19 '^ 3 ) . The process is best initiated by a broad (but brief) description 
of the major features of a youth work training program, highlighting its 
impl:^ed and explicit goals followed by a logical "translation” of those 
goals into more specific criterion categories within which variables ana 
scales can be defined. The measures desired should be as objectively 
specsjfiable as possible and should also draw upon experience from previous 
rese^^rch in which similar criterion constructs have been applied. 

1 . Characteristics of a Youth~Work Training Program 

Major features of a youth -work training program from which categories 
of objectives can be defined for the analyt.ical purposes of this study are 
to be based on a brief description of the Neighborhood Youth Corps, Out-of- 
School Program (Federal Register, 1969). This full-time, nonresidential 
program may be considered sufficiently representatives, in bk>th the enrollee 
population served and in its broad fimctions , to meet present study needs. 

-1C for any youth-work training program, (and as previously indicated) 
the : erall goal that provides the basis for program design is the enhancement 
of enrollee employability. To accomplish this intent of improving the 
i_'-ll-a.u’ial^s capability to obtain and hold a Job, the local sponsor of a 
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NeighlDorbood Youth Corps Project (NYC) must provide a setting in which 

school dropouts receive guidance and counseling services and an opportunity 

to work, toT at least a minimum wage, on some form of "suitable” job. 

A work supervisor who is an employee ot the organization in which the 

enrollee receives his work-training (often a governmental agency) maintains 

an informal relationship with the NYC project through a project guidance 

counselor who is professionally responsible for the progress of the enrollee. 

Other than the requirement for vocational guidance counseling and supervised 

work experience, there have been no rigidly defined prograjn components, 

* 

curriculum materials or specific technical skills to be acquired. 

Although there is no formal program termination point (i.e., "graduation" 
or "completion"), the nominally prescribed period of enrollment is con- 
sidered to be one of six months. An enrollee may of course leave the program any 
time he chooses. The enrollee population served is, by definition, from a 
poverty-level background (based on family income) and composed of school 
dropouts who have generally been unable to obtain either suitable or regiilar 
employment on their own. Age ranges have varied since the inception of the 
program but have remained roughly within the adolescent age levels of l4 
to IT years . 

The structural aspects and functional requirements of the program lead 
to reasonably e.xplicit ard identifiable sets of assumptions , or intentions , 



* 

Formal requirements for remedial training in verbal and arithmetic 
skills were first instituted in the summer of 1970 for all NYC projects under 
the NYC II Program. Althougji many projects had undertaken such skill 
training on their own prior to that time, it was purely voluntary (often 
under specially funded studies) and often temporary in its availability 
to the enrollees . 
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concerning specific objectives that are held by program administrators and 
professionals. For example, it is generally assumed that the guidance and 
counseling services provided are instructional with regard to vocational 
possibilities, training requirements, the value of improving academic 
standing (e.g., return to school or formal education), job seeking skills, 
(e.g., how and where to look for jobs; appropriate interview behaviors) 
and proper on-the-job behaviors, (e.g., acceptance of supervision, getting 
along with co-workers, coming to work on time). In addition, the guidance 
may extend to personal problem areas and overall social adjustment outside 
of the work setting (e.g., f ami ly problems, legal difficulties, and handling 
of finances). All of these are closely complemented by paid, on the job, 
experience intended to enhance enrollee work— related adjustment, familiarity 
with job requirements , organizational expectations and in some cases 
(depending upon the type of work experience) development of specific tech- 
nical skills . 

The combination of counseling and work experience are viewed, in turn, 
as the means for instilling personal confidence in the enrollee regarding 
his ability to function in a work setting and in his desire to achieve 
vocationally — the effects of which woxild, hopefiilly, ramify to all areas 
of social and vocational adjustment. 

From such a broad constellation of program characteristics and assumptions 
it is possible to postulate sets, or ’^categories , of general outcomes along 
with specific variables and scales that might define those categories. 

2 . Description of Criterion Categories and Measures 

It should be evident from previous discussion that for an initial 
examination of youth-work training program criteria it is desirable to specify 
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sets of criterion categories along with a fairly vide range of associated 
outcome variables, that are both logically justifiable and reasonably 
practical to obtain. Such variables can be considered xinder tvo major 
headings based on their applicability at different points in time. One set 
deals with outcomes that define hov veil the enrollee fared at, or near, 
the ** completion" of his participation in the program, while the other 
criteria are applicable, largely, to post-program periods during which 
longer-term effects on enrollee performance could be expected to occur. 

There is a degree of appropriate overlap in a number of the criterion 
categories and variables at these two time periods (e.g. , those dealing 
with personal-social adjustment areas). Other categories are of necessity 
unique (e.g., all aspects of post-program full-time job experience since 
they can only apply to enrollees who have left the program). The specific 
variables and scales applied to each category are referred to under "data 
sources" using their numbered designations in the appropriate criterion 
questionnaire.* 

Program Completion Criteria : representing short-term outcomes consist of 

the following categories and associated vajriables used in their measurement: 
a. ■ Vocational planning ability : can be considered intrinsic 

to the counseling and guidance component of a youth-work 
training program, particularly that aspect of guidance 
which provides information concerning available occupations 



The Enrollee Program Completion Questionnaire is shown in Appendix B 
and the Post-Program Qx^r^stiormaire in Appendix C. Evaluations by counselors 
and work supervisors are presented in Appendix D and an Employer Evaluation 
Scale in Appendix F . 
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and -the steps necessary to achieve realistic 
vocational goals. This category ot enrollee per- 
formance capability deals with vocational plans 
formulated at the time the enrollee is "completing’* 

NYC and the ways in which he expects to implement 
those plans. Willingness to plan, the extent of 
planning and the degree of reality are basic 
•theoretical concerns in vocational guidance and 
have been considered in the research literature as 
important to all vocationally-or iented behaviors, as 
well as to the eventual degree of Job success achieved 

(Stephenson, 1955s Super, 1957)* Variables in this 
category are intended to reflect vocational planning 
intentions of the enrollee, his overall concern for 
various job characteristics in formulating those plans, 
and the "quality" of his planning. In line with the 
desirability of criterion variables being as objective 
as pos sible , the enrollee ' s planning intentions require 
responses that are as behavi. orally oriented as possible. 
Data s our ces-— cons 1st of Post— Program Que st ionnair e items 
17, 19, and 20 to 26 ; each scored as separate variables. 
Item weights for l8(a) through l8(g) are summed to provide 
a single "Awareness of Job Characteristics" scale score, 
b. Feelings of vocational adequacy : is a category intended 

to deal mainly with the enrollee ’ s explicit perceptions 
of his chances for success as a potential employee. This 
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atti-tudinal construct can te considered as a 
reflection of the degree to which ]the enrollee 
feels he can "make it” in a work setting after 
he leaves the training program (i.e., vocational 
self-confidence or readiness)- Rational hases for 
the choice of the construct stem from specific 
intentions hy counselors and work supervisors to in- 
still feelings of vocational adequacy in the enrollee, 
through guidance that apprises him of his skills and 
capabilities and through work experience that offers 
familiarity with a job setting as well as the chance 
to develop proper work habits and skills - The concept 
is probably related to the more widely used ”self- 
esteem," or self-concept dimensions (lA^lie , I961) that 
have also appeared as scales in various studies of 
vocational development among disadvantaged adolescents 
(Freeberg, 1970; Hunt &. Hardt , I966 ; Walther & Magnusson, 
1967); in theoretical formulations of career choice (Super, 
1957 ) ond in studies of adolescent vocational adjust- 
ment (Holland, 196'U) - However s the scales uud items 
chosen here, are intended to be a more direct and 
“objective” reflection of intended behaviors for youth 
work-tradning program outcomes than the often vaguely 
defined and. diffuse self-concept measures customarily 
applied as predictors . 
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Da,ta SQ-urces — consist of the items of a"Vocat ional Adequacy" 
scale -with item weights summed to provide a single scale. 

The scale is made up of program completion questionnaire 
items 10 to l6 with items 20 and 20(a) also representing 
aspects of the same construct. 

c .. Per s onaH.— s oc i al a d.justment ; represents a criterion category 
with hi^ "face validity" or logical relevance. It is 
considered particularly desirable per se , hy program 
professionals » to assist the enrollee in achieving a 
level, of social adjustment and personal satisfaction that 
allows him to function adequately in a Job, avoid legal 
difficulties and achieve some degree of personal satisfaction 
in his family life. In defining information needs for 
w::.-k -training programs i two primary areas of personal-social 
adjustment have generally been considersd of importance. 

One has been designated as "community adjustment" (U.S. 

Dept, of Labor, 1969 ) and is considered to be reflected 
in delinquent behavior as measured by involvement with the 
police. Logically, however, this concept can be extended 
further and touch upon personal-community adjustment 
variables that concern Involvement with other members of 
the community, as well as the individual's health status 
and financial planning or monetary responsibility. A 
second major adjustment area for consideration deals with 
the enrollee ' s relationships with his family, particularly 
the extent to which they are a supportive rather than an 
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obstructive influence on his efforts to improve. It 
has been suggested that patterns of family responsibility, 
demands and encouragemen influence critical aspects of 
vocational, academic «md social pex'forinance among dis- 
advantaged youngsters (Gordon, 1965)- 

Data souD'ces — include questionnaire items 32 to 3^ as the 
variables representing family adjustment and 35 to Ul as 
the co 33 imunity adjustment variables . In addition, data on 
police contacts were obtained, when available, from local 
law enforcement agencies as an external check on the 
enrollee— provided information . 
d. Work -training program adjustment and capability : is 

intended to represent a category of external criterion 
judgments (as opposed to enrollee self-evaluation) * adicative 
of the degree of overall adjustment and success achieved 
as a result of participation in the youth-work training 
program. The customary criterion measures used for this 
purpose consist of formal ratings by those who are in a 
position to make appropriate observations of enrollee 
performance. Several such evaluative sources are readily 
available from a youth work -training program. One is from 
program professionals ^usually guidance counselors) and 
the other is from those responsible for the training and 
supervision of the enrollee in the work setting (i.e, , 
a work superv’isor ) . A third potentially useful source 
is peer ratings, for which no systematic research 
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applications are known with youth-work training program 
populations . Ibis type of criterion measure has been 
shown to be ox* value for a variety of training programs 
(Wherry & Fryer , ‘19^9) an<i appears worth examining in this 
situation. There mi ghi:^ however ^ appear to be greater 
difficulties than usual in obtaining peer ratings from 
disadvantaged adolescent groups. Among those are an 
unusual degree of reluctance to m.ake specific evaluative 
Judgments about fellow enrollees and difficulties in 
obtaining a sufficient sample of enrollees, well enough 
acquainted with one another in the work training setting, 
to provide meaningful ratings . 

Data sources — are based on three rating scales: (a) 

An eleven ‘(ll) item coxmselor rating scale used for both 
male and female enrollees, (b) Two, ten-item. Work- 
Supervisor scales— —one for male enrollees and one for 
females — with several items differing on each of those 
two scales (Appendix E) . The Counselor and Work Supervisor 
scale items were, derived from scales used in a previous 
study of NYC enrollees (Freeberg, I 968 ) and (c) Two types 
of Peer Rating scales (Appendix B ) that were necessitated by 
variations in the nature of rater-ratee acquaintance . One 
set of these items could only be xitilized for those peers 
whose acquaintance with the x*atee had been in the NYC 
project and/or on a social basis, while the other set is 
applicable to those whose acquaintance is only social (i.e.. 
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outside of NYC exclusively) . Since it was not practical 
to obtain criterion data for all enrollees at any given 
project, those who did appear in the sample were asked 
to choose two fellow enrollees to he rated. This was 
seen as a means of increasing the odds for obtaining at 
least one rating on as many enrollee respondents in. the 
sample as possible . 

e. Work mot i vat i on : could probably be considered one of the 

more widely agreed-upon standards of enrollee behavior, 
applicable to any broad definition of "success” at program 
completion. The counseling and work experience components 
of the program clearly emphasize this motivational outcome, 
Enrollee expressions of a desire to succeed vocationally — 
as indicated by willingness to take specific actions (e.g., 
job training, searching for employment) — constitute one of 
the means . possible for measuring such a criterion category 
at program completion. Scores obtained from scaled items 
that dealt with general feelings about the importance of 
obtaining and holding a job have shown modest correlations with 
coTonselor and work supervisor criterion ratings for NYC enrollees 
(Freeberg, 1968). However, in order to test a more objectively 
definable index of motivation, the measure to be utilized 
here as a criterion is intended to go beyond generalized 
attitudes about employment and deal with specific, althou^ 
hypothetical, opportunities to make vocationally- and 
education ally -related decisions, in a "simulated” choice 
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situation^ where a particular joh heing sought has a 
high degree of desirability for the enrollee . 

In addition, althou^ there are ±ew if any objective 
behavioral measures at program completion that reflect 
performance motivation directly, two pieces of admin- 
istrative information were considered to be of 
value for examination as motivational outcomes. These 
are absences from job site(s) and the number of job site 
changes made by the enrollee during his enrollment in NYC. 

Data sources — are the job lists on page 10 of the questionnaire , 
scored in terms of total number of jobs selected. Items 4U 
and 45 were scored as separate variables and items 46(a) 
through 46(g) utilized summed item weights to obtain a single 
"Work Motivation Scale" score. 

Enrollee absences from the work site(s) and the number of 
work site assignments were obtained from the NYC project 
records, with values adjusted for length of program enroll- 
ment in the scoring of these variables. 

Post— Program Criteria : represent the longer— range outcomes 

% 

that might logically be affected by participation in the youth— work 
training program. (As will be apparent in the descriptions below, certain 
of the criterion categories and their associated variables can, of necessity, 
apply only to former enrollees with employment experience; others are equally 
applicable to those with employment experience and those who never held a 
job after leaving the training program. ) 
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a* Extent or level of employment : is the category that 

encompasses informataon undoubtedly considered of 

primary importance in determining the degree of success 
for a youth -work training program, over any post-program 
period- Measures that define the extent to which the 
enrollee eventually achieves productive, remunerative 
employment would possess a high degree of rational 
validity if their value were to he based on the 
professional Judgment of work-tr aining program experts . 

In addition, the relevance of these criteria to longer- 
term vocational achievement might be expected to pz'ove 
significant, sinc€i early job performance tends to be 
one of the better predictors of future job success c 
Generally, criteria dealing with the nature and extent 
of employment have been a cornerstone fcr a number of 
Vocational development theories that depend for their 
verific -.'ttion on measures dealing with occupational 
choices, job advancement, sequence of jobs held, 
dura.' . ..n of stay, etc., (Borow, 1964). Similar vocational 
performance variables that are considered applicable (and 
available) as youth-work training criteria would include: 
the extent of employment since leaving NYC (i.e., whether 
enrollee held, or now holds, a full-time job); length of 
stay on job(s); hours worked per week and an index of the 
Equality” of the jobs held based on a combined rating of skill. 
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status level, and advancement possibilities. In the pres- 
ent study, derivation of a composite ”Job Quality" score 
for the most recent or presently held Job vas based largely 
on information from the National Opinion Research Center Job 
status scales (Duncan, Hatt , & North, I 961 ) , the Dictionary of 
Occupational Titles (U.S. Eiaployment Service, 19^5 ), and the 
Occupational Outlook Handbook (U,S. Dept, of Labor, 1969 )- 
Data s ounces — consist of responses to the Post-Program 
Quest ionnaire for items 11 through I 6 and item 23? each 
scored as separate variables. Job quality vas assigned 
an appropriate veight of "1," "2," or "3" based on the 
guidelines for this categorization shown in Appendix D. 

It was also intended that the information be verified — 

(where practical or available) — from USTES and/or social 
secvirity records . 

b . Work stability and w ork performance : cover what should 

be logically related categories involving the capability 
to retain a Job over some period of time (stability) and 
to perform successfully in a work setting. Assessment 
of on-the-Job performance has long constituted a major 
category of outcomes in industrial training research 
(Burtt, 1942; Ghiselli & Brown, 1955; Tiffin & McCormick, 

1958 ) with the number of different indices , customarily 
utilized to specify this construct , ranging from unit 
work-output measures and "creative" achievements , to 
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proficiency ratings ty others (e.g., supervisors). 

Those first two forms of criterion data are relatively 
impractical to obtain for the enrollee population of 
interest in this study so that ratings become the 
primary source of outcome infomiation concerneo. with 
Job proficiency. The job proficiency score can be 
obtained from ratings made by a present (or former) 
employer and would of course represent a variable 
with a high degree of logical relevance as a youth- 
work training outcome. 

Job stability as a measure has also been considered 
relevant for a number of occupational research programs 
(Holland, 1964; Fames, Mil jus , & Spitz, 1969; Super, 

1957 ), but its role among criteria for youth-work training 
program is relatively unknown. Specific variables that most 
conveniently define the measure are the number of 
held during a given post-program period, the length of 
time employed on each job and the reasons for job termination. 
Data Sources — for these categories consist of questionnaire 
items 16 , 17 , 17 a, 23, 24, 25 and 58 , each scored as 
separate variables. A four-item Employer Rating Scale is 
presented in Appendix F and yields a single score consist- 
ing of the sum of the four wei^ted items . The brief 
scale contains several item types that have generally proven 
effective in employer assessments (Owens & Jewell, 1969 ) • 
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c. Vocational ad.justment ; should serve as art important 
adjunct to the high priority Job performance outcomes. 

An increasing amount of vocational guidance research 
has come to focus on the personal adjustment category,, 
by dealing with measures of the individual's perceptions 
of his work environment and the ways in which the work set- 
ting interacts with his values and needs (Dunnette, Campbell, 

& Hakel, 196T ; Herzberg, Mausner, & Snyderman , 1959) « Some 
empirical basis for the potential relevance of variables in this 
category exist, since measures of Job satisfaction appear 
to have moderate effects on worker productivity and Job 
stability (Brayfield & Crockett, 1955; Vroom, 196U). 

Although often difficult to specify as a measure because 
of its multi fact or composition, determination of worker 
satisfaction has nevertheless obtained strong support over 
the years as a useful form of outcome measure (Dunnette, 

Campbell & Hakel , 1967 ; Heron, 195^; Hoppock, 1935; Whei-ry, 
1958). Problems in measurement, stemming from a need for 
more precise definition of the characteristics of Job sat- 
isfaction have stimiilated a number of attempts to identify 
the factors that comprise the construct (Kendall, Smith, 

Hulin & Locke, 1963; Sedlacek, 1966; Wherry, 1958). How- 
ever, uncritical and direct application of items from many 
of those factored scales to disadvantaged adolescents may 
be questionable since, in most instances, they are based on 
samples of white-collar professionals, semi-professionals 
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and clerical workers. Fortunately, Hinrichs (1968) has 
recently derived a set of joh satisfaction factors for a 
wide range of industrial personnel with differential 
factor patterns reported for different employee levels. 

That information allows for the selection of scale items 
somewhat more appropriate to the Johs that former NYC 
enrollees would he likely to enter. With some changes 
in item wording, nine items for a Joh Ad,justment scale 
were derived from the factors presented in Hinrichs' study. 

Data source — is the nine item 'Joh Adjustment" Scale utilizing 
items 26 through 34 of the Post-Program Questionnaire which 
yields a single score based on the sum of the item weights. 

Item 22 is considered somewhat similar to Joh satisfaction, 
hut is stated in terire of the extent to which vocational 
experiences in general have met the enrollees expectations 
and is treated as a separate criterion variable, 
d. Personal -s ocial adjustment : is a category for which variables 

were selected on the basis of essentially the same rationale 
as presented for the Program Completion Questionnaire. How- 
ever, some additional personal adjustment variables can be 
considered as more logical, and/or available, after the enrollee 
has left the program. One is the variable of number of visits 
to a physician, which has had widespread conventional use 
as a criterion measure indicative of personal adjustment, 
family mobility (i.e., number of residences since leaving 
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"tile work-training program) and use oi* Tinancial 
credit f*or making purchases. 

Data sources — f*or those enrollees who hold, or have 
held 5 full-time employment (Post— Program Questionnaire: 
Section l) consist of items 8 ^ 10 and 43 through 55 
which are all treated as separate variables • 

For those enrollees who have never been empj.oyed full— 
time^ the adjustment variables (in addition to items 8 
and lO) are to be found in items 28 through 38 of Section 
II; with all items treated as separate variables. 

Number of police contacts were obtained from local law 
enforcement agencies as representative of conmrunity 
adjustment . 

e. Work-seeking motivation and planning ; are based on a 
rationale similar to that provided for the Program Com- 
pletion Questionnaire. However , in the selection of 
specific variables to define these categories, a number 
of additioncTl experiences during the post-program period 
(i.e*, work experiences) can be used as more objective 
measures of the enrollee*s willingness to seek employment, 
the degree to which he perceives work as a worthwhile pur- 
suit and his ability to plan for his vocational future =. 

As will be evident from the specific variables chosen, 
the motivational and planning categories are assumed to 
overlap considerably since much activity that involves 
vocational planning also implies a strong motivational 
component. 
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Motivational outcomes are to te reflected in such 
measurable activities as: whether or not the enrollee 

is currently working full time or has held a full-time 
job, the amount of additions,! training or formal 
schooling he has obtained, the length of time after 
leaving NYC before he found the first jobj the number 
of sources utilized in finding the first job and the 
reasons for leaving jobs. For those former enrollees who were 
not previously employed, there are potentially useful 
motivational variables dealing with attempts at further 
schooling, stated intentions to seek employment and the 
degree of concern with various characteristics of jobs. 

This ,tter form of measure is presented as a "Concern 
for Job Characteristics" scale that incorporates content 
customarily used in construction of job-satisfaction 
s cales . 

Criterion variables dealing more specifically with 

vocational planning skills ax*e derived from enrollee 

estimates of how high in status and salary he expects 

to go in the present job, what plans he has for the 

relatively near future — including his longer range occupa— 

"tional goals — ^how he plans to achieve the intended goals 

(scaled in terms of planning "appropriateness" and "qxiality") 

and the extent to which he has saved money since leaving NYC. 

Data sources — for motivational variables in Section I of 
< 

the Post-Program Questionnaire are derived from items 11 
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IL5 , 17 (d.) and I 8 through 21; each scored as separate 
variahles . The closely related planning shills variahles 
are obtained from items 35 through 42 , each of which are 
scored separately. 

For those enrollees who have never been employed (Section 
II of the Questionnaire), the applicable variables for 
motivation and planning consist of items 11 through 27 
scored separately and the sum of item weights for items 
25(a) to 25(g) to form a single scale score ( i . e . , a 
”Cc . iern for Job Characteristics” scale). 

B. Questionnaire Design and Administration 

The purposes of the study and the use of disadvantaged adolescent 
school dropouts of relatively low verbal skill levels as the respondent 
groups dictate the need for special attention to criterion questionnaire 
format, content, wording and method of present at ion - 

The two separate questionnaires shown in Appendices B and C 
were designed to deal with the two major types of outcomes under 
study — i.e., the more immediate, or proximal criterion variables relevant 
at the point of program completion and those that might serve as longer- 
term, or int ermediate, ob j ect ives following program completion. Use of two 
relatively distinct sections of the Post-Program Questionnaire, however, 
stems from the need to obtain information about former enrollees who have 
never held employment and cannot, of course, be queried regarding important 
criterion measures that deal with job-related experiences. Indeed, for 
criterion development purposes — especially in devising criterion, measures — 
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-bile former enrollee group 'fch.a'b has never worked since leaving "bhe 
•training program is, in many ways, more like those completing the program 
than those who have had full-time employment experience. 

Format and Content : A basic ground rule in the design of the 

ciuestionnaire was that the content and format for items defining a similar 
Criterion category would, where logical and practical, he as similar as 
possible. (This improv'^s the possibilities of gross comparison between 
variables and criterion dimensions) . Such an intention : ; ^ for practical 
purposes,, highly limited when comparing program completion and post-program 
samples but is possible to a greater extent for the two subgroups of former 
enrollees. Even there, however, such comparisons are almost entirely 
limited to the categories of personal-sociah^- family adjustment and those of 
vocational planning-motivation. 

The first seven demographic items of each questionnaire are intended 
not only for identification purposes but to serve as a potential check 
on the interviewer’s veracity in obtaining data from the respondent (i.e., 
the demographic infoirmation of the questionnaire should match information 
on an enrollee personal data form to which the interviewer would normally 
not have access). Questionnaire verbal leve.l is intended to be appropriate 
for adolescents with relatively low vet'bal skills and couched in a style that 
allows for continuity in interviewer presentation with a natural flow from 
item to item. Thus, complete sentences are utilized throughout and the total 
questionnaire is designed to serve as am orally prer;entable ’’script . In 
line with this intent , items that cover similar subject m.atter and items 
of all coherent scales (e.g., the Vocational Adequacy S 
'n seqi^ ...je. 
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Admin i s t ration : Item presentation is intended to "be oral with 

the interviewer reading all item material to the enrollee . Marking of 
responses was to be done by the interviewer,, preferably, but could have 
been carried out by the enrollee if the interviewer considered it desirable, 
or the enrollee requested that he be permitted to do so- Response prompting 
was allowed only where indicated on the questionnaire. The interviewer was 
to read the introductory remarks accompanying the questionnaire and obtain 
all of the information at a single session extending over approximately 30 
to ho minutes. As indicated in the introductory remarks a all respondents 
were paid for their participation. 

C- Sample and Data ‘Collection 

Questionnaires were administered to a total of 379 male and female 
respondents. Of these, 13T»who were enrolled in the NYC Out— of —School 
Progi’am for at least several months , constituted a "Program Completion" 
criterion sample. The remaining 2k2 were former NYC enrollees who had been 
out of the program for o- period of h months to approximately 1 1/2 years.- 
One hundred and fifty-four of these former enrollees had full-time employment 
experience (Post— Program. ^ Employment Experience Sample) and 88 liad never worked 
full time since leaving NYC (Post-Program:No. Employment Experience Sample) . 
Full-time employment experience was defined, arbitrarily, on the basis of the 
ex— enrollee ’ s having worked more than 20 hours' per week for at least one 
week- These three subsamples are treated separately, for the most part, 
throu^out the subsequent analyses. 

Former enrol.lees -who had entered the military -were excluded from the 
study sample, since it was felt that the social adjustment and Job-related outcome 
measures of the questionnaire were hot entirely applicable to military 
personnel . 
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The 379 respondents of this cross-sectional sample were enrolled, or 

formerly enrolled, in Neighborhood Youth Corps projects in six (6) cities 

* 

■with me’tropoli’taxL-area populations in excess of one-q.uarter million. 

Three of these -orhan areas are in the northeastern U.S., one in the 
southeast, one in the mid-west and one on the west coast. These cities 
are indicated hy letter codes and geographic locations in Table I, along 
with the distribution of the enrollee sample by sex and criterion group. 

Six trained inter'vi ewers ? who were residents of the urban area in which 
they conducted the interviews , were responsible for obtaining question- 
naire data. All six were professionals or semi— prof essionals with experience 
in educational and training settings and none were employed by a 3 ^?eighborhood 
Youth Corps project at the time of data collection. The HYC projects 
agreeing to participate, served as the- basic inf ormaT .ion source for identifying 
present and former enrollees , their last known addresses and/or phone numbers 
and also provided counselor and work supervisor ratings . 

The information obtained from, or about, these respondents was gathered 
largely diuing the slimmer of 1970 5 from approximately June through early 
October . 

Special Problems 

It is a truism that information availability is a primary determiner 
of evaluation and research capability. In that regard, certain characuer- 
istics of a youth-work training program and of the poverty-level population 



%y agreement with various NYC regional administrative personnel and 
project directors, neither the specific cities nor the projects are 
identified. 
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served, can generate critical p robleras in obtaining desired criterion 
information and in the utility of the information obtained. 

(1) Data gathering - constitutes one of the major areas of concern, 
since there are serious limitations in the quality of 
information obtainable from poverty-level , largely minority- 
group, adolescents vho live in urban ^etto areas. Follow- 
up of former youth-work, training rrogram enrollees over any 
extended post-program period can prove particularly difficult 
and costly. There are usually no formal, or informal, links 
routinely established between the program and the enrollee 
once he leaves and, certainly, there is no compulsion for 
him to participate in data gathering efforts. Distrust and 
avoidance of "establishment" agencies on the part of minority 
group members often adds additional difficulties to problems 
of locating former enrollees and enlisting their cooperation. 

An appreciation of many of the problems and some of the 
techniques for data gathering from, "hard-to-loc ate poverty- 
level, irrban groups can be gleaned from a collection of papers 
dealing with that 'topic (U.S. Dept, of Labor, 1969)* The 
overall impression ■f'rom those papers, and the investigators' experi- 
ences in the present study, is that one can locate a large proporti 
jf individuals in almost any minority ' group or poverty -level 
population if there is a willingness to pay what amounts to 
"private detective fees." Unfortunately, such a requirement 
would not he likely to prove feasible for extensive and 
continuing research and development efforts, so that it is 
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often necessary to settle for less than the specified 

sample and a reduction in the scope of information ohtainahle. 

Some mitigation of the problem is usually sought in greater 
dependence on data that do not retpiire the enrollee's or ex— 
enrollee's participation (i.e., "external" sources). Both 
intuitively and on the basis of a need for measurement 
accuracy, it appears more desirable to obtain such "objective 
oi external, outcome information, since it can serve as criterion 
data in its own right as well as a check on the more "sub- 
jective" (spT -P-report ) forms of information provided by a 
respondent. Customary sources of external data include 
employment records from state agencies, job performance 
data, credit agency files, records of local law enforce- 
ment and welfare agencies, schools, etcj as well as evaruations , 
or other information from family and friends. But, this does 
not necessarily provide a wholly satisfactory solution since 
a number of the previously mentioned data gather: problems 

remain and new ones arise. Thiis , there is not only considerable 
effort requi.red to obtain information, from, say, police and 
w- If are agency records, but the data are often of imperfect 
quality (U.S. Dept, of Labor, 1969) — especially in terms of 
completeness and comparability between urban areas. In 
addition, ratings by employers, friends, or family can be 
as difficult to obtain as informaticn directly from the former 
enrollee and subject to as. many biases. 
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More serious problems are introduced ^ 3:. cvever 5 in terms 
of ethical implications and tne possibility of detrimental 
effects resulting from the collection of data about an 
ex-enrollee without his knowledge and consent . The fact 
that some forms of desired information may be in the public 
domain does not entirely negate the concerns. For example > 
one of the most desirable forms of criterion information 
regarding a former enrollee is probably his record of on— 
the-~job performance j which is only obtainable by contacting 
present or previous employers . Atteupts to do this can 
arouse reasonable apprehension on the part of many respondents 
that any seemingly official, or quasi-official, inquiries 
may Jeopardize their status with their employer . 

There is no question being raised here regarding the value, 
for study purposes , of sources of criterion information 
external to the respondent’s self-report. The present study 
has attempted to incorporate such forms of information 
where feasible within the time limitations, financial 
constraints and estimated accessibi3,ity , or adequacy, of the 
data available from various external sources, bather, the 
intention is to amplify the point that the "right to privacy 
is a legitimate contemporary social concern. A number of 
issues stemming from that concern remain to be dealt with as 
part of the broader of problems serving to impose 

serious limitations on the. development and application of 
objectives essential to proper research and evaluation of youth- 
work training programs. 
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(2) Sampling biases - vith relatively •unknown effects on sample 
composition and findings can *be imposed by both the data 
gathering difficulties described above and various aspects 
of NYC prograri- operation. As would be the case for any post- 
training program follow-up study, there is the perennial 
difficulty of estimating whether the individuals who voluntarily 
participate are significantly different — in "meaningful” 
traits — from those who cannot be located, or choose not to 
serve as respondents. Demonstration of the similarity of the 
located, volunteer respondents to a more complete sample 
on the basis of available ex- post facto variables (age, sex, 
education, length of stay in program, etc.) does not neces- 
sarily demonstrate that selection biases failed to affect the 
scores on a n-umber of post-program, dependent (criterion), 
variables. Selection biases of mvmerous sorts are still likely 
to produce serious undermatching, or lack of representativeness, 
in s^ich at-tempts (Greenwood, 19^x) • 

Since there are- no reasonably complete samples available of 
foiiner youth-work training enrollees measured on a sufficient 
number of pertinent outcomes, it remains difficult to estimate 
^ the extent of the biases introduced with any given enrollee or 
ex-enrollee sample obtained. The samples utilized in the 
pre ’)tudy do not, by any means, qualify as "reasonably 

complete." Considerably less than full participation was 
acliic ^ed for the respondent groups sought. For example, among 
th se pi*esently enrolled, the percentage of "successes" 
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reported by the interviewers (i.e., respondents participating 
based on number of attempted contacts) was about 60 ^ to 
8 ’^% depending on the uirban locade and, perhaps, equally on 
interviewer capability and the degree of cooperation by a 
given NYC project staff. 

Among former enrollees , who had been out of the prograiu 
for several months and more, the drop-off in those percentages 
is considerable since the interviewer is faced with 
the major problem of first locating the former enrollee. 

Here, the overall success rate reported was as low as 30 % 
to 35^, although in two urban areas this was reported as 
somewhat better than 50 %. As might have been anticipated, 
the interviewers reported more problems in locating those 
ex-enrollees who had been out of the NYC program for longer 
periods — particularly those who had been out for more than 
one year. Clearly, the greater the data gathering limitations 
the greater the sample biases likely to have been imposed. 

One further problem in sample bias , introduced as a result of 
the composition of the NYC enrollee population in urban areas, 
is the relatively small proportion of males who either enter 
the program or remain enrolled for periods of several months or 
more. This enrollment imbalance has been characteristic of 
many types of youth-work training programs during the past 
several years and is further compounded as a biasing effect by 
greater difficulties — which interviewers -uniformly reported — 
in finding male ex-enrollees aiid obtaining questionnaire data 
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from them. The female former enrollees were simply easier to 
locate and more willing to cooperate as respondents* Sample 
composition hy sex is shown in Table 1 5 where the problem of 
obtaining male samples of sufficient size can be seen to 
result 5 even though there were specific efforts by the inter- 
viewers to obtain a larger ninnber of male respondents. 

Despite the sampling difficulties outlined above ^ it is 
felt that the prasent sample can be considered suitable 
in terms of sizes geographic distributions and other enrollee 
characteristics 5 for purposes of this initital study 
aimed at identifying potential criterion variables. 

D . Data Analyses 

In order to examine the value of a variety of criterion measures, in 
the broad context req^uired here, correlational analyses would appear to 
provide the maximum amount Of coherent information. By determining the 
patterns of relationships among the variables and their underlying factors 
or dimensions, some bases are provided for choosing measures with the hast 
potential for structuring given categories of outooiries. 

Initially, identification of the dominant dlmenBions can be used (a) 
as a check on the extent to which the logical, ^ priori^ criterion categories 
(previously specified) are empirically supportalsle and ("b) whether other 
categories j not previously liypotheslzed ^ appear to he worth strengtheiilng . 

A second step in utilizing the analyses would he to specify the most useful 
dlmensioiis and varlahles for future application. GThe ground rules for 
accomplishing this are not only based on the strength of a given variahie 
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(i.e .5 its loading on a dimension) but also on "rational priorities" 
assignable to objectiv-es of a youth-work training program. For example, 
those work-related variables that are considered of especially high pri- 
ority (e.gv, job quality, salary^ length of employment, employer rating, 

etc.) “=whera they show strong cluster^ — would constitute the dimensions 

i 

of primary interest, Attitudinal types of variables (salary expectations. 
Job satisfaction 5 etc.), that aore also present on those same dimensions, in 
t'LU’n become stronger candidates for continued use in a criterion system. 

It also follows that lower priority variables, which make up relatively 
small isolated dimensions, are those likely to be considered as having 
lesser value for future use. 

The basic analyses to be performed consist of computing the inter- 
correlation matrices for (a) 32 criterion variables chosen for use with 
the NYC Program Completion Sample of 137 enrollees , (b) 36 variables 
applied to the sample of 154 former enrollees who had full-time employment 
experience (Post— Program: Employment Experience Sample) . and ^ (g) a third 
matrix with 27 variables applicable to the sample of 88 enrollees who 
had never been employed full time (Post-Program:No Enplojnnent Experience 
Sample). Each of the 3 matrices were factor analyzed using a principal 
components solution (l,00*s in the diagonals) with varimax rotation to 
orthogonality of the factors extracted (Kaiser, 1957). Missing data analyses 
were used throughout in computing the correlations for the matrices. 

As part of the necessary examination of frequency distributions , for 
determining which outcome variables were suitable enough to retain in the 
analyses 5 it was also possible to make general judgments about potential 
sample biases for nmle and female subsamples and, at the seme time, to 
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present certain descriptive findings about former enrollees that were 
considered worth highlighting for their programmatic implications. 

III. KesiQ,tB and Discussion 

The intercorrelations of the criterion variables and their resulting 
dimensions are best considered under the classifications of short-term 
(proximal) criteria applicable at program completion and longer term 
(intermediate) criteria for post-program use/ SepM'ate analyses are 
required in the latter group for those ex--enrollees who have had full-time 
employment experience and those who had never been employed full time. 

Missing data occurred throughout the soiipla for most variables ^ because 
of one or a combination of several reasons. First , the information about 
the enrollee may not have been readily available. For example ^ employer 
ratings were obtainable for only 37 former enrollees in the sample of 15^ 
since present or former employers coiild not be contaetedf they refused 
to provide a ratings or the enrollee did not grant permission to contact 
the employer (as occurred for approximately of the sainple). Second ^ 

respondents may have chosei:. not to provide the information requested by 
the interviewer (e.g., an outright refusal to do so, or the belief that 
they did not know the answer with sufficient ae curacy) « For still other 
variables , missing data are unavoidable since the response is contingent 
on a previouB item that can only be answered by a subsample of respondents 
(e.g,^ -’Reason for leaving last Job” can only Include a subsample of the 
Post— Program Baiployment Experience group who had held more than one Job). 

Not all criterion variables that appeared in the questionnaire were 
foimd suitable for analyses . A relatively small proportion of them were 
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dropped because of e^remely poor distribution characteristics , totally 
inadequate sample mlze^ or excessive numbers of response inconsisteneies 
that resulted from misinterpretation* 

A* PrQ^ajn Completion Criteria 

Table II prasants the 32 x 32 intercorrelation matrix obtained from 
the NYC Program Completion group. Significant correlations ^ indicated in 
the table by asterisks ^ are found to be uniformly low to moderate in size 
with approximately 13^ of those sigiificant correlations reaching the 
*05 confidence level 5 or better* 



Certain of the individual correlations stand out as reference points 



within the questionnaire* The most striking of these relationships is the 
r of *79 between the enrollee*s questionnaire response to the amount of 
trouble experienced with the police and the actual oecurrance of police 
contacts (Variables #20 and #28) . At that level of relationship the 
enrollee ’ s self— report of trouble with the police mighty in essence, serve 
as a reaBonable substitute for the actual occurrence of police contacts 
especially where the latter is too difficult or costly to obtain. On the 
basis of this correlation it could also be assumed that the anrollee’s 



In order to assist the reader in associating the variables analyzed 
in the matrices with the questionnaire items from which they were obtained 
the numbered variables for each of the three respondent groups are listed 
in Appendix A with the associated questionnaire item number. 



worth noting as indicative of overall response consistency 
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willingness to provide an accurate response, for such a supposedly "sensitive 
form of informations would spread to overall reeponue accinracy for other 
quest lonnaire i-|eniG . 

Another correlation that reflects a degree of qiiestionnaire internal 
consistency is the substantial r of between the "starting salary" 

expected on the first joh and the "highest salar^y" that the enrollee expects 
to achieve on that same Joh (Variahles and #8) , High correlations between 
variables that should logically be related (e,g ,5 different questions about 
salary expectations) help to confirm response consistency. Similarly, it 
can be noted that no clear instanees occur among the significant correlations 
in the matrix that would constitute illogical relat^" unships or lead to 
suspicions of response inconsistency , 

An additional relationship of note is found between Goimselor Ratings 
and Work Supervisor Ratings (Variables #26 and #2T)* The correlation of 
,^4^0 obtained here is vary similar to one of .36 previously obtained for 
such raters evaluating NYC enrollee s Borne two years prior to collection 
of the present data (Freeberg^ 1970), Thus j a degree of confidence can 
be placed in the stability of thoee two important external criterion measures 
It is 5 however, the pattern of relationships and the criterion clusters, or 
dimensions, formed that best summarise the Information of interest in the 
matrix and provide some basis for choosing the most usable outcome measures , 

GrlterlQn dimensions i, Seven factors^ aocoimting for 47^ of the total 
variances were extracted from the 32 x 32 matrix of ProgrMi Completion 



Interviewers were not to ask permission to cheek police records nor to 
inform the enrollee that they intended to do so. 
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criterion measures. The choice of seven factors for rotation was based on 
a decision to deal with one more than the approximate number of criterion 
categories hypothesized (i.e. , those six having been; Yooational Planning^ 
Peelings of Vocational Adecjuacys Pamily-Personal Adjiistment, Community Adjust- 
ment, Training Program Perfoiroaiioe and Work Motivation). Five of the seven 
factors were found to be sufficiently Interpretable ajid to have value as 
broad outcome dimensions. Loadings ^ on the five rotated factors interpreted, 
are presented in Appendix G, These five factors are discussed below along 
with a listing of the variables that possessed loadings of adequate size to 
enter into the factor interpretation (loadings of .30 or higher). 

Two of these factors that are relatively dominant , based on accountable 
variance, deal with somewhat distinct forms of adjustment that are designated 
a-s Training Program Adjustment (Factor I) and Social-Community Adjustment 
(Factor II ) . 

Factor I 

Training Progrsuii Adjustment 

Variable Loading 



20 


Amount of Trouble with Police 


.66 


27 


Work Supervisor Ratings 


-.64 


30 


Number of Work Site Absences 


.61 


26 


Counselor Rating 


-.57 


28 ' 


Actual Poll ce Cont act s 


.54 


22 


Number of Jobs Chosen for Training 


-.49 


IT 


Save Money 


-.31 


9 


Ways to Look for First Job 


-.30 
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Factor II 



Social Adjustment 



Variable 



Loading 



l8 Number of People Giving Enrollee Hard Time 

Ih Get Along with Family 

28 Actual Police Contacts 

13 Ability to AccompliBh Plans 

20 Amount of Trouble with Police 

31 Peer Rating Score 



.71 
= .6o 
-56 

.^7 

-,41 



The first factor defines a relatively broad aspect of adLjustment , after 
some period of NYC enrollment * that is concerned largely with performance 
in the program but also possesses a major component of outside adjustment in 
the fojm of problems with police. ^us ^ the primary loadings of this bipolar 
dimension present a pattern of lower work supervisor and counselor ratings 
aecompanied by more work^-site absences and more trouble with the police 
(admitted by the enrollee and actual). It also incorporates tendencies by 
the enrollee to indicate fewer Jobs as acceptable for training, when a number 
of choices are presented., a lesser likelihood that he saves money from his 
NYC pay and poorer knowledge of relevmt ways to search for a Job that he 
would desire. 

Factor II is more clearly one of Social- Commimitiy Adjustment ^ primarily 
outside of the worki-training program, with the enrollee perceiving more 
people in the community as giving him a "hard time," having more trouble 
with his family, as well as with the police, and also being rated lower in 
overall adjustment by his fellow enrollees . As in the prex^ious factor, 
there is a tendency for the maladjusted enrollee to show up as poorer on 
some aspect of vocational planning. In this case he tends to indicate fewer 
relevant ways for accompllahing hie stated vocational plans . 
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A third relatively clear snd dominant dimension that emerges is 
definable as Job Aspiration Level (Factor III} and is based almost entirely 
on the enrollee ’ s vocational expectations and assessment of his capabilities. 

Factor III 

Job Aspiration Level 

Variable Loading 



8 


Highest Salary Expected 


.73 


T 


Stajrting Salary Expected 


.69 


2 


Quality of Job Desired 


.58 


12 


Long Range Job Plans 


• 56 


13 


Ability to AocompliBh Plans 


.kj 


1 


Vocational Adequacy Scale 


.36 


25 


J ob Mot i vat i on S c ale 


.30 



The higher expectations of this enrollee cut imiformly acroBS salary 
expectations^ the quality of the job he desires vhen he leaves NYC^ the level 
of Job he plans to obtain over the long run and the relevance of hip stated 
plans for achieving his goals. As is logical ^ the factor also incorporates 
stronger feelings of vocational adequacy and a higher degree of Job motivation 
in the face of obstacles . 

Bomevhat close to this third factor coneeptually , but empirically 
distinct^ is the rather minor factor designated aa Planning Competency 
(Factor V). This fairly clear dimension is Indicative of the individual 
who would choose a Job for which he feels he has adequate knowledge and ability, 
and who demonstrates competence in Imowlng how to look for the Job he wants. 
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Factor V 

Planning Competency 



Variable 




6 Ability to Do Job Sought ,72 

5 Itoowledge of Job Sought .72 

9 Ways to Look for First Job 

11 Job Search Plans (Suitability) .35 



The coherence of the factor, despite its relative specificity^ makes it 
appear vorth expanding for futiire criterion development efforts by use of 
additional similar items and scales that demonstrate specific vocational 
planning skills * 

The final interpretable factor (IV), although not as readily designated 
as the others, seams to identify the enrollee who possesses (and projects) 
a Self-Confident Tma ge . 



Factor IV 

Self-- Confident Image 



Variable Loading 



32 . 


Humber of Times Chosen for Rating 


.60 


31 


Peer Rating Score 


.49 


4 


Avareness of Job Characteristics 


.4? 


26 


Comiselor Rating 


.4l 


1 


Vocational Adequacy 


.4o 


19 


Health Problems 


-.38 


11 


Job Search Plans 


-.32 



He is better known by his peers (i.e*, selected more frequently for 
rating) rated hi^ier by them, as well as by the guidance counselor* In 

addition, he tends to show more interest in the characteristics of any Job 
he would seek; to be confident of his vocational adequacy (ability to 
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obtain and hold a job) ’^hen he leaves NYC and to see himself* as being in 
good health. Interestingly^ however, he tends to show poorer planning skill 
if he shoiild be faced with loss of the first Job after NYC. For this reason 
and the fact that the work supervisor is the only rater not influenced by 
him in a positive way (i-e.^ no interpretable positive loading for on-the- 
Job proficiency ratings), there is some suspicion that the concept of "image’- 
appropriately enters into the desi^iation of the factor. 

It should be evident that effective potential criteria for defining 
enrollee "succesB," after a period of program participation, could be derived 
from a number of these different dimensions — -several of which bear a degree 
of similarity to hypothesized categories of outcome (e.g.. Training Program 
Adjustment, Social Development). Such empirically defined dimensions 
should also be stable enough to lend themselves to the formulation of new 
criterion variables that can be used tc broaden their applicability. On the 
basis of the present evidence, where variables are needed for particular, 
immediate uses (e.g., test validation or construction of the moat effective 
scales or composite criteria)} the end-of-program objectives of the sort 
likely to be most effective can be def*ined as : 

1. Objeotlve-External Data (sources other than enrollee self-report) 
. Work supervisor and guidance coimselor evaluations 
. Trainee absences during enrollment in the program 
. Difficulties ea^erienced with law enforcement agencies 
. Evaluations by peers 
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2, En^ollee Provided Da^a ( obj ect Ively irer Ifiable ) 

, Assessment of* probleiiiB witb law enforeement agencies 
. Monetary planning (e.g., savings) 

* Appnopi^iateness of* Job decieions and Job ehoices (i.e, ^ 
planning) fon the immediate and longer term 

3 * J^rollee Provided Data ( attitudinal^ not objectively verifiable) 

^ Feelings about family (acceptance; cooperativeness ) 

. Perceived problemB with members of the conmiunity 

, Feelings of adequacy or ’’readiness" for employment 

, Assessment of personal capabilities with regard to future 
Job performance 

Post —Prog ram Criteria (Enrollees with full-time employment experience) 
Thirty— sxx ( 36 ) variables 5 covering outcome measures for the group of 
former enrollees who have held full-time employment, were intercorrelated 
and are presented in the rnatrix showi in Table IIX. These longer-term 
(intermediate) criteria that incorporate a niamber of the important work-re- 
lated performance outcomes result in matrix that yields the largest pro- 
portion of si^ificant correlations (approximately 22$ of the 630 r’s) 
and a more clearly definable set of factors than t#he other two criterion 
groups under study. 

Evidence for overall internal consistency in enrollee response to 
questionnaire items is demonstrated again, as for the Program Completion 
data, in the high correlation of .83 between "amoxmt of trouble with the 
police reported by the enrollee (Variable # 30 ) and "actual police contacts" 
(VajTiabie #36) • In addition, high levels of correlation that are logically 
expected and indicative of response consistency are seen in the substantial 
r of between the "number of Job applioatiom filed" (Variable #13) and 




m: 



'Vrltii Full-Time Itoployment Experience] 
(II = 151))^ 



= 62 - 
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"number of places interviewed" (Variable #13) and in the r of .50 between 
the "Job Satisfaction" score (Variable #19) and the enrollee's assessment 
of the extent to which he has "met job expect: at ions " since leaving NYC 
(Vajrlable #l4). 

Again, for this matrix as, for the previous one, there are no signifi- 
cant correlations that are illogical in their Interpretat j.on and. no reason 
to suspect general reBponse inconslsteiicy for the q^uestionnalre . 

Criterion dlmensionB . Seven factors accounting for 50% of the variance 
in the 36 X 36 matrix were extracted and rotated to orthogonality by the 
methods already Indicated. These were found to represent, adequately, the 
dominant dimensions of the matrix with six of the seven factors considered 
sufficiently interpretable to be utilized for defining criterion constructs 
(Appendix C). Ihose six possess an overall, although limited, GOrraspondence 
between their designations and the logically derived post-program criterion 
categories hypothesized previously in Section II, Thus, Factor I, General 
Job Success and Adjiistment , appearing as the dominant factor- -with the largest 
va,riance accounted for and the greatest nimiber of factor loadings at levels of 
^30 or greater— is concerned with overall adjustment that has a distinctly 
vocational focus. 
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Paetor I 

General Job Success and Adjustmen-t 
Variable ' Loading 



3 


Employed Now (No/Yee) 


.80 


2U 


Family Feelings about Enrollee 


, 68 


35 


Employer Rating 


.59 


29 


Save Money 


.57 


7 


Length of Stay on Job 


.52 


lU 


Met Job Expectations 


,U8 


36 


Police Contacts (Actual) 




k 


Industry Category (T^Jhite/Blue Golleo") 


-.36 


33 


Number of Visits to fiSS 


-.35 


30 


Amount of Trouble with Police 


-.32 


3h 


Permission for Employer Rating 


.30 


18 


Job Promotion (No/Yea) 


.30 


19 


Job Satisfaction 


.30 



The factor can, be seen bo characterize bhe enrollee with full-time 
employnient experience who is also currently employed at the time of the 
Interview. He receives a distinctly higher employer "rating, has stayed 
on his Job longer, has been more likely to get promoted on that Job, 
be more satisfied with it and, in addition, feel that he has come closer 
to meeting his job expectations. He is also more likely to be einployed 
in a white-collar industrial setting and to give permission to obtain an 
evaluation from his employer. In addition to those aspects of vocational 
success, thes e performance outcomeB define an individual who sees his family's 
opinion of himself as a favorable one, saves money regularly, has less trouble 
with the police and is less likely to utilize the State Employment Service 
(undoubtedly because he is less likely to need its services ) . in essence, 
then, a major dimension useful for defining post-program success should be 
conceived of as more than one dealing with Job performance alone, important 
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as remains , but as on© 'that incorporates other community and 
personal adjustment behaviors as well. 

Wliere a more narrowly defined. Job performance scale is desired 5 the 
variables of Fae"bor II, Striving for Vocational Success^ form a dimension 
that should be considered^ 



Factor' II 

Striving for Vocational Success 



Variable I oading 



18 job Promotion (No /Yes) .6l 

26 Number of People Giving Bnrollee Hard Time ,58 

IT Amoxint of First Raise .55 

1 Nmnber of Interviewer Contacts ^,^2 

2 l Job Quality Bought J+6 

3 k Permission for Employer Rating .hO 

5 Job Quality ,38 

35 Employer Rating » 3 k 



The positive enrollee attributes for this dimension are defined on 
the basis of more Job promotions and salary raises, a higher q,uality of 
emplojnnent obtained and a higher employer rating of Job proficiency. But 
there are variables external t o t he Job per se , that also add to a description 
of the former enrollee who is striving for job suceesB. Thus, he was found 
to be more difficult to contact for an interview and he also saw more 
people as "giving him a hard time" at work and In the commimlty* (Perhaps, 
not an unexpected perception for a poverty level individual who is trying 
to get ahead, vocationally, and having some success in accomplishing that 
goal . ) 

Similarly Job-specific In its pattern of loadings is Pactor III, Job 
Stability_-Mobility ^ which defines a pattern of enrollee Job behaviors that 
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entail the filing of more applications.^ before obtaining the first post- 
program Job and , correspondingly 5 having had more Job interviews. In 
addition, there have been more visite to USTES-^—probably resulting from a 
greater need for its services — along with greater mobility vocationally 
( "more Jobs held since leaving NYC") and geographically (lived in more places). 
This general mobility and "Job hopping" can be seen to carry a negative 
connotation for a sample of disadvantaged yoi,mg adults , since the enrollee 
also tends to be rated lower in Job proficiency by his employer , 

Factor III 

Job Stability -Mobility 

Variable Loading 



13 


Number 


of 


Applications Piled 


.82 


12 


Number 


of 


Plac es lilt ervlewed 


.80 


8 


Number 


of 


Jobs Held Since NYC 


.57 


33 


Number 


of 


Visits to USES 


.kg 


2 


Number 


of 


Places Lived 


.46 


35 


Employe 


r Reting 


-,Uo 



If Factor I could be viewed as defining overall job suceesB and adjust- 
ment for those whose present employment tends to be in white— collar work 
settings 3 then Factor IV can best be interpreted as one of general vocational 
adjustment for those former enrolleeB whose employment experience has been 
primarily in blue-collar Industry , 
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Factor IV 



Blue —Collar Job Suceess 



Variable 



Loading 



10 Time to Find First Job 

6 Hours Worked Per Week 

15 Starting Salary 

22 Level of Long-Term Plans 

k Industry Category (White/Blue Collar) 

16 Humber Salary Raises 

5 Job Quality 

19 Job Satisfaction 

23 Get Along with Family 

7 Length of Stay on Job 



56 

5U 

hj 

h2 

4l 

k± 

ho 

38 

35 

35 



The pattern of loadings for this factor of Blue-^Collar Job Success^ 
depicts the former enrollee who found his first Job after HYC more quickly, 
worked more hours per week, received a higher starting salary and more raises, 
in a better quality Job at which he tended to remain employed longer and 
with which he expressed greater Job satisfaction* However, this successful 
Job performance pattern is also marked by poorer long-term Job planning. 

The only social adjustment variable loading on the factor is at a modest 
level (-35)? indicating that positive feelings about getting along with the 
family play a role in defining success for a blue-collar worker. 

Within the realm of personal social adjustment , there are two dimensions 
that define relatively independent areas of outcomes. One is limited almost 
exclusively to a Coiniiiunity— Family Adjustment pattern (Factor V) and the other 



The result may be largely artifactual since the future plans of the 
sucGessful blue-collar employee mi^t lead him to continue doing what he 
has been doing successfully. Such a response on the questionnaire was 
scored toward the lower end of a four point scale. Higher scores were 
assigned to responses that involve looking for a higher level job or going 
to school, which responses are probably more typical of white-collar employees. 
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depic*ts a form of adjus'tmen'fc Is essentially vocat ional—^1 • e . 

Vocational AdJ\istment and Satisfaetion (Factof VI), 

Factor V 

Coimrrunity and Family Adjustment 



Varia'b le 

25 

30 
36 
32 
28 

26 
11 
23 
21 



Financial Assistance to Family .6l 

Amount of Trouble with Police -*60 

Actual Police Contacts -^.59 

Ease of Obtaining Credit ^U8 

Himaber of Times Visited Loctor ^45 

No. of People Giving Enrollee Hard Time - . 3U 
Number of Sources Used for First job 
Getting Along with Family . 32 

Job Quality gou^t (Long Term) .32 



Factor VI 



Vocational Adjustments Dissatisfaction 



Variable 




Loadin 


9 


Reason Left Job (Poor /Good) 


.68 


20 


Level of Short-Term Plans 


.42 


21 


Quality of J ob Sought 


.4a 


23 


Get Along with Family 


.39 


35 


Employer Rating 


.37 


28 


Number of Times Visited Doctor 


-.36 


14 


Met Job Expectations 


-.35 


29 


Save Money 


.33 


16 


Number of Raises 


.32 


19 


Job Satisfaction 


-.30 



Factor V can be seen to load positively on family iteDis that deal with 
providing financial assistanee and getting along with the family in general. 
As part of this pattern^ there is better community adjustment in the form of 
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less trouble with the police , being able to obtain eredit more easily 
and feelings of fewer people giving him a hard time in the eoirammity or 
at work. This seme ex=enrollee also tends to have seen a physieian more 
freq^uently since leaving the NYC programs a practiGe which, among poverty 
level groups , appears to be associated with positive attitudes and social 
adjustment. (This represents a reversal of the role of this criterion 
variable usually found for middle-class and military populations.) Further* 
mores in obtaining the first Job the adjusted enrollee requires the assistance 
of fewer institutional or community sources and tends to choose a Job of 
higher quality that he would seek over the longer term. 

Although the positive pattern of loadings on Factor VI ( Vocational 
Adjustment”^DisB atisfaction ) are indicative of a toTm of Job success ^ the 
interpretation of its accompanying negative loadings contain an element of 
obvious Job dlssat is faction . Thus ^ the factor defines an ex^enrollee who 
presents good reasons for having left any Jobg and of having been sun employee 
who obtained more raises and was also rated higher in proficiency by his 
employer. His short-term planning skills are superior as are his desires for 
a higher quality Job for the future. Some non* vocational forms of adjustment 
are seen in his claim of getting along with his fOTiily and of saving regularly. 
However 5 as part of the pattern , this former enrollee does not feel that he 
has met the Job expectations that ho had when he left the NYC program and he 
also tends to be more dissatisfied with his Job, Interestingly, in light of 
its loading on the previous factor, the measure "frequency of physician visits" 
loads negatively here (i,e, , less likelihood of visiting a physician) and may 
be seen as consistent with the dissatisfaction or maladjustment aspect of this 





dimension 






On tile basis of the above analysis ^ optinium variables lor potential 
use in constructing a set of post-prograjn objeetives should be drawn from: 

1. Objective- External Data (sources other than enrollee self=report ) 
. Rating of profieieney by present or fo:i^aer employers 

- Difficulties eixperieneed with law enforcement agencies 
. Ease with which contacted for an interview 

2. Enrollee Provided Data (objectively verifiable) 

^ Ge ogr aphi c mob il ity 

. Utilization of USTES 

. Current employment status and type of Job 

- Ease of obtaining employment (time to obtain employment ; 
ntmib er of att empt s re quir e d ) 

p Assessment of problems with law enforcement agencies 

. Salary level (present and past Jobs) 

, Job performance (promotions and raises) 

• Job stability (number of Jobs and lengt;h of Job stay) 

• Monetary behavior (savings and use of credit) 

3. Enrollee Provided Data ( attitudinal : not objectively verifiable) 

- Job related feelings of personal a at is fact ion 

- Perceptions of family role 

- Perceived problems in commimity at work 

. Level of Job plans and expectations 

As a tentative look at possible equivalence between criterion dimensions 
for the short-term program completion outcomes and those of the longer-term 
post-program period ^ it is possible to compare the two sets of factors by 
gross V'* al inspection. Granted the caution that the two matrices have 

O 
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only B. minor portion of their variables in common and utilise different 
members for these cross- sect ion ally obtained samples g it s of interest to 
note a degree of similarity in the dominant "succeBs" factors found for the 
two samples* Each success factor wash&aidly loaded on similar Job performance 



on variables of police contacts, difficulties with peoj>le in the community 
and the extent to which the enrollees were getting along with their family. 
Further 5 but somewhat more speculatively, a form of dimensional equivalence 
might be assumed for the "aspir ational" grouping found with presently enrolled 



marks those who have left the program and achieved a measiore of Job juccess. 

In any event , such eontrasts must remain pure conjecture until they can 
be confirmed by direct measurement of criterion equivalence over time, 
feasible only with a longitudinal study sample. 

^ Program Criteria (Enrollees with no full-=-t ime employment experience ) 

Resiolts achieved by the analysis of outcome variables for a post“program 
enrollee group, that had not worked full time since leaving the training 
progrujfi, should be viewed as fairly limited in their applicability. Not only 
Is the available sMiple size relatively small (N - 88), but it is a sample 
composed primarily of female former enrollees. Males who had remained 
unemployed for months or years after leaving NYC were exceptionally 
difficult to locate and interview. In addition, the natiare of the outcome 
variables potentially applicable to such a group are narrow in their scope 



variables (e.g*, ratings by work supervisors for one group and by employers 
for the other), accompanied hy similar adjustment variahles (e.g., police 
contacts, savlr?g of money). In addition hroad similarity could be inferzed 



from the major "adjustment" factor of each sample with regard to loadings 



trainees, and the "striving for vocationaJ, success" behavior pattern that 
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and degree of* oId jectivity . On a purely raisional ‘basis, it is difficult 
to define specific outcomes for a group that has never held fuH^time .jobs . 

As indicated previously, erlteria for such a sample hear a greater resemblance 
to outcome measures applicable to a currently enrolled (program completion) 
group than to a post-program group 'with employment experience. In 
addition they lack even the external evaluative possibilities offered by 
coimBelor, work supervisor and peer ratings reasonably obtainable for a 
currently enrolled group - 

One alternative lb to eliminate this post-program group as inappropriate 
for outcome measurement once they are identified as not having had any post- 
program employment experience (i,e-, that they have simply "failed'" to 
achieve measurable benefit from the program) . Such an alternative has , in 
effect, been rejected for this study on the basis of arguments against use 
of a single outcome variable as the sole eveiluative standard. It also seems 
somewhat impractical to assume, without fiju^ther evidence, that no beneficial 
effects of prograin participation accrued to such a large proportion of a 
former enrollee population or that any possible benefits derived are not worth 
the measurement attempts. Thus, at this stage of the eriterion development 
process , it is seen as valuable to consider the 27 potential outcome variables 
available for analyses with this post-^program sample. 

The 27 X 27 intereorrelat ion matrix of Table IV contains a number of 
si^if leant correlations that can be pointed to as reflecting a degree of 
internal consistency for the q^uestionnaire responses. For example, the corre^ 
lation of similar magnitude (r = ,79) is, again, foamd between the enrollee 
report of amount of trouble with the police (Variable #23) and interviewer 
data on actual police contacts (Variable #27). Logical, internal consistency 
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Post-Pfogram Criteria 

(EriTollees 'witti Mo Full-Time Employment Experience 1 
(li = Ba)^ 
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^Becimal points omitted 

jjjMissing data occurred thr'oughout the matrix: l'*s range froin approximately 30' to 88 » 
r significant at ,05 confidence le^el or tetter 






can be inferred from 'what should be, and is, a high correlation (r = .T9) 

between the relevance of reasohs given for picking a job that is sought 
(Variable #7) and the relevance of reasons given for having the ability to 
perform that job (Variable #10), Similarly, there is a degree of consistency 
shown by the r of .65 between the level of long range work plans (Variable 
#l6) and the level of response appropriateness regarding the means to achieve 
those nis (Variable #17). The one inconsistency found among the significant 
correlat j.ons is the r of between "number of places lived" (Variable #2) 

and "nirmber of interviewer cont acts required" (Variable #1). Its contribution 
to the factor patterns will be seen below to offer little clarification for 
the unusual result . 

Criterion dimensions . Seven factors were extracted from the intercor- 
relation matrix, as in the two previous analyses, and rotated by the same 
procedure. The seven factors aeeounted for 30 $ of the total variance with 
only four of these considered interpretable and possessing loadings of suffi- 
cient number and magnitude to warrant discussion of their potential value 
(see Appendix G). As might have been anticipated for this ssanple, with rela- 
tively few criterion categories that can be hypothesized (i.e., primarily 
categories of motivation , planning and adjustment), there were correspondingly 
few usable factors and even those were found to be relatively we^ and diffi- 
cult to interpret. 

Two of the dimensions deal with forms of adjustment ,cne being concerned 
primarily with adjustment in the eoamnunity and the other taking the form of 
a desire for personal improvement. 
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Factor 1 

Coiiimunity Adjustment 



Variable Loading 



2T Actual Police Contacts -.85 
23 Amoijuit of Trouble with Police - . 8U 
25 Ease of Obtaining Credit .52 
19 Future Financial Contribution to Family .46 
12 Highest Salary Expected —.36 
22 Nijuriber of Times Visited a Doctor .35 



Factor’ II 

Striving for Personal Improvement 



Variable 




Loading 


5 


Part-Time Work Experience (No/Yes) 


.81 


11 


Starting Salary Expected 


.59 


1 


Number of Interviewer Contacts Redb^ired 


.39 


3 


Level of Current Activity 


.31 


18 


Get Along with Family 


.30 


20 


Number of People Giving Enrollee Hard Time 


-.30 



Factor I^ Community Adjustment ^ describes a pattern of outcomes for 
an enrollee who has fewer problems “^iih the police ^ claims more ready access 
to credit , a willingness to make a greater financial contribution to family 
income (if he shoiild be employed In the future) ^ an expectation of a lesser 
starting salary and a tendency to visit a doctor more freq.uently since 
leaving the program (the latter result being consonant with the previous 
finding that the variable loads positively on an adjustment dimension.) 

Factor 11 ^ Striving for Personal Improvement ^ has a factor pattern 
more indicative of personal adjustment that describee the enrollee who has 
heild part-time employment and^ probably as a result, expects a h:igher starting 
salary on the first full-time Job.. He also tends to be engaged in a form of 







higher level activity at the time ot the interview (i*e., "in school’* 
or "in another work training program" as opposed to "still Jot hunting 
027 "not working, not looking") , to he getting along hettei' with his tamily 
and to perceive fewer people in the coinmimity as giving him a **hard time. 
Factor III can be interpreted, tentatively, as an aspiration or con- 
fidence dimension, with aspirations tempered by "reality** of expectation. 

Factor III 

Realistic Aspirations 



Variably Loading 



9 Ability to Lo Job Sought *17 

^ !Kfumber of Places Lived ^66 

8 Knowledge of Job Sought -6^ 

12 Highest Salary Expected “"59 

19 Future Financiixl Contacts to Family .55 

18 Get Along with Family 
25 Ease of Obtaining Credit 

11 Starting Salary Expected -.32 

1 Humber of Interviewer Gontaets Re^uir- d -.32 



This factor of Realistic Aspirations is defined by the high positive 
loadings on feelings of ability to do a Job sought and the possession of 
knowledge of what it takes to accomplish that job. The **reality" of the 
aspirations, for a group that ha^s never he3,d employment, is Inferred from 
the aocompanying lower expectations for starting salary on a first job and 
for highest salary likely to be achieved. Some aspects of adjustment are 
also present In the form of willin^ess to contribute to family income, 
getting along with the family and the ability to obtain credit . Present 
in the pattern is an indication that the enrollee has been more mobile since 
leaving NYC (lived In more places) but, somewhat unexplainably , tended to 
be contacted mere readily for the interview. 

O 
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The laiS'b faQ'tor, Job Mo'bivatlon ajid Planning (Factor IV), is "the least; 
dominant hut most readily interpretahle of the four, with positive loadings 
on concern for Job characteristics, the level of short term vocational plans, 
the level of the enrollee’s current activity and the quality level of the 
Job that he seeks. The coherence of this factor, despite the relatively 
few variables that enter into its definition, mark it as a candidate for 
further expansion, by attempts to find other positively associated variables 
of similar content. 



Reconnuended outcome variables for use with this sajiiplev which are fairly 
self- evident from the factors described, would consist of: 



1. QbJeotlve^Externad Data (source other than enrollea self-report) 
. Difficulties experienced with law enforcement agencies 
2- Enrollee Provided Data (objectively verifiable) 

. Assessment of problems with law enforcement agencies 
. Part--time work experience 

. Ciirrent activities (training and Job hunting) 

. Ability to obtain credit 



Factor IV 



Job Motivation and Planning 



Variable 



Loading 



15 Concern for Job Characteristics 



10 Reason for Ability ( Irrelevant /Relevant ) 
18 Get Along with Family 



U Level of Short-Term Plans 
3 Level of Current Activity 
6 Job Quality L evel Sought 



.76 

^6T 

.65 

.53 

.31 
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Enrollee Provided Data (Att itudlnal : not objectively verifiable) 



. Level of Job and salary expectations 
- Perceptions of Job performazice capa.bili't ies 
, Fajni ly ad J us t inent- 

In general 3 the factors described for this group are sufficiently weak 
and unclear to leave open the q^uestion of whether there is an adequate range 
of variables that can serve as acceptable outcome measures for those who had 
participated in a youth work training program but were subsequently unable 
to obtain employment. Some extension of the factors and the variables 
identified above may improve the precision of the measures and the coherence 
of the dimensions (e,g.5 use of more extensive scales and of other external 
evaluations such as ratings by peers or family) - Nevertheless ^ on the basis 
of the present evidences existing criteria would have to be shown to possess 
reasonable equi'’''alence over time (longitudinally) s with a larger sample than 
was utilized here, before it can be concluded that there are multiple criteria 
Justifiably applicable to this particular subgroup of former enrollees , 

Of necessity, all of the recommended sets of outcome variables presented 
above , for the three enrollee and former enrollee subgroiips , are limited by 
the range of variables that entered into the analyses , Other epecific forms 
of outcome measures, if obtainable and incorporated, might have resulted in 
modifications to the dimensions identified — ^though hopefully leaving much of 
their basic meaning and interpretation intact, (New dimensions would, of 
course, be expected to emerge had specific intellectual or remedial skills 
measures been considered appropriate or practical for inclusion as criteria 
at this stage of development.) However, given the relatively wide range of 
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behavioral oulGomes u‘biliz;ed5 the factors should be seen as no more than 
coherent groupings of broad object , -s ^ within which certain classes of 
variables can be specified for youth- work training program evaluative needs , 

D* ” lhe” Employment Criterion ^ Because of having dealt with the two former 
enrollee groups separately in the above analysis , there has not been an 
opportunity to consider Employment Status per se No Employment 

Experience vs. Employment Experience) as a separate variable 5 in terms of 
its relation to other outcome measures . Although not basic to the criterion 
development purposes of the present s-^udyj it is of value to ask what 
relationships this ^'highest priority*^ variable might have with those con- 
current outcome measures logically common to a full-time employment experience 
group and a non -work experience group ^ 

Fourteen such comparable variables are available from the post-program 
questionnaire items used in the present study and are 5 of ccurae , limited 
to variables derived from adjustment and motivation— planning categories . 

The partial correlations between Employment Status and the scores on each 
of the lU variables s with ^'Length of Time Out of NYC ” partialled out (i.e,, 
held constant, statistically), are showi in Table Y* 

The only significant correlations with Employment Status occiu/ for 
’*Quality of Job Sought-’ (r = ^Uo), indicating that those with employment 
experience tend to seek a higher level Job than those who have never been 



Tills should be seen as differing in intent from the more conventional 
research use of the employment status variable, which is to accept it, 
rationally, as the ’’best” outcome and then to search for enrollee personal 
background characteristics or training pro^am eharaeteristics that differentiate 
significantly between groups who have and have not held employment (i,e,, the 
variables most predicti%''e of the employment outcome) * 
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Table V 

Partial Correlations^ of Post—Program Hhuployment Experiences (No/Yes) 
vith lU Criterion Variables for Post— Program 
Employment Experience Group 
(Former Enrollee Sample; N S 2 U 2 ) 



Criterion Variable 



Partial r 



1 


Nxmiber of Interviewer Contacts Required 


-.01 


2 


Number of Places Lived Since NYC 


-.06. 


3 


Quality of Job Sought 


.40 


U 


Long Range Plans 


.04 


5 


Getting Along with Family 


-.or. 


6 


Amount of Financial Help to Family 


.15 


T 


Number of People Giving You a Hard Time 


.04 


8 


Health Status 


.03 


9 


Number of Times Seen Physician 


.01 


10 


Trouble with Police 


- . 06 


11 


Use of Credit 


.04 


12 


Trouble Obtaining Credit 


-.10 


13 


Visits to State Employment 


-,06 


lU 


Police Contacts 


-.05 



i^^Length of time out of NYC program" held constant (i.e., partialled out) 

w - 

^^r significant at .05 confidence level 
r significant at .01 confidence level 




'P': 



88 






employed fiill time and "Amount of Financial Assistance to Family" (r - *15) 



which const! tutsF i very slight tendency for those with work experience to 



he willing to contribute more money for family use ^han those who have never 



worked would be willing to contribute (if they were to obtain employment)* 



Based on the relatively limited number and scope of outcome variables 
analyzed here (that are common to former enrol_Lees with end without employ- 
ment experience), there is no strong evidence of a meaningful- pattern of 



concurrent relevance, for the Employment Status criterion measure* It should 
be clear that a considerable loss of information would be incurred regarding 
what^ has happened to former enrollees and, more important, how and why it may 
nave happened if there were total dependence on this relatively limited, 
dichotomous, criterion in attempting to understand perfonriance outcomes* 

E. Some Descriptive Highlights of the Post-Program Samples 

Although the main concern of this study is one of defining important 
criterion dimensions that stem from relationships between outcome variables , 
examination of the distributions for these variables leads to a number of **survey,^* 
or descriptive, results that seem worth highlighting. These descriptions 
ar'e summarized very briefly and only as suggestive of research hypotheses , 
or further study needs , regarding post-program enrollee behaviors that might 
have been influenced by program operations* 

(l) Of the sample of 2h2 former IJYG enrolleee (selected only on the 
basis of whether they were out of the program for approximat ely S to l8 
months), 88 enrollees or 36% were found never to have held full-time employ- 
ment* Add to this the finding that, of the I 5 U who held full-time employment 
in at least one Job, h3% were unemployed on the day of the interview and it 
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becomes apparent that these school dropouts did not fare particularly 

iu achieving steady employment during mid— 1970.* 

(2) Of the 154 enrollees who eventually found full-time employment 
37% reported doing so ■within 30 days after leaving WYC . 

(3) BTYC served as the primary source through which these 154 obtained 
their first full-time job (27$), with "friends" reported as the next most 
frecxuent source (l6^), 

(4) Between the time they left WYC and the time of the questionnaire 
interview, two-thirds of the sample of 154 with full-time employment experi- 
ence had held only one Job, with a mean time on the Job of approximately 

5 months. In effect, they could not as a group be classified as "job hoppers" 
although it is equally likely that they have lesser opportunity than other 
workers to move from Job to Job. 

(5) For the 154 enrollees with full-time employment experience, 50^ 
claimed never to have utilized the State Employment Service; whereas, among 
the 88 without full-time employment experience, 6l% indicate never having 
visited SES. The State Employment Service appears to be avoided as a resource 
by a majority of former enrollees. Along with this, it should be noted that 

only 11 ^ of the sample attributed the obtaining of their first Job to the 
efforts of SES. 

(6> Over half of the sample of 88 enrollees, who had no post-program 
employment experience, still evidenced some vocational "motivation" either 






Unemployment levels reported for the black teenage 
about that same time, rajiged as hi^ as 34^ (Manpower and 
Education Weekly, 1970). 



population , 
Vocational 



at 
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in the rorm oif active Job seeking ( 37^)3 or by enrollment in another youth- 



the sample (22 enrollees ) found to be "not working — not looking," with 
10 of those 22 classified as "housewives," 

In conjunction with the above descriptive characteristics for single 
criterion variables, it is appropriate to point out^ briefly^ the possible 
biases that may have operated for several of the criterion measures with 
regard to sex. These samples, if divided by sex, were far too Inadequate 
in size to allow separate correlational and factorial analyses. However, 
sample sizes in two of the three samples (Program Completion and Post-Program: 
Employment Experience Groups) were adequate enough to permit gross comparisons 
of the dietrlbutions of scores for males and females on most variables. 

With onJ.y a few exceptions , the male and female subgroups were found 
sufficiently comparable on the distributions of scores selected for analyses 
to warrfi^t their combined use. Those criteria retained for use because of their 
high level of logical priority as outcomes, but for which biasing effects for 
sex can be suspected, were : (l) The variables of ’police contacts" and 

"trouble with police" (i.e,, claimed and actual) in all three samples — with 
females having significantly fewer police contacts , by far, than males and 
consequently a much more highly skewed distribution. The extreme skewness found 
led to dichotomization of the variable for analytical purposes (i.e.. No Police 
Contacts vs. Police Contacts); ( 2 ) "Starting Salary" for first Job obtained 
( Post^Progr am : Employment Experience Sample) showed a significantly higher mean 
for males with greater positive skewness of the distribution for females* 

( 3 ) Salary expectation scores ("Starting Salary" and "Highest Salary Expected") 
in the Program-Completion Sample also resulted in the female sample having 



working training program or in a school (23^) v There were, however, 25^ of 
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dis-tributioiis of* greater positive skeTmess and significajitly lower mean 
values; (U) ’^Number of* Jobs Cbosen for Training^” in the Program Completion 
group showed a more negatively skewed distribution and higher mean value 
for the female sample. 

A look at any additional suspected biasing sources 5 expressed as subgroup 

difference in distributions or other characteristics of the variables (e*g. ^ 

regression slopes ), and the detailed analyses required to confirm those 

biases ^ would require more extensive research effort than was practical 

* 

within the scope, intent and available data of the present study. 



IV . Conclusions and Reconmiendat ions 

This criterion study phase has represented an initial attempt to provide 
information that would aid in the selection of objectives potentially applicable 
to a youth^work training program^ At the same time, it was intended that the 
study illustrate an analytical approach to criterion development based on the 
use of multiple criterion measures. By utilizing eamples of adolescent school 
dropouts , some of whom were enrolled in a youth -work training program and 
others who were former enrollees , it has been possible to identify an assort- 
ment of empirically defensible and coherent factors , or categories of 
objectives, from which measures bast applicable for research and evaluation 
purposes can be identified. In essence, the faetorB have provided an overview 



Determination of bias or "arnfairness" for any behavioral measures has 
taken on much more complex meaning than implied here (Cleary, 1968 ; Rock, 
1970), Such efforts entail detailed regression analyses in a predictive 
framework to demonetrate more specific operation of subgroup criterion biases. 
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oT structural features for a broadly --based set of ”eore" criterion measures. 
They therefore serve as a framevork for assessing the value of the particular 
measures applieds ets well as being suggestive of scales into which new criteria 
can be incorporated* At the same tlme^ the factor designations and inter- 
pretations have been imavoidably descriptive of important patterns of behav- 
ioral outcomes that characterise what has ^ in fact , happened to samples of 
enrollees who are (or were) enrolled in the.‘ Neighborhood Youth Corps work- 
training program* 

Certain assumptiohs_, which were advanced for ‘imdertaking the development 
of program criteria^ have been viewed as integral to the design and method 
of the study and bo the utility of the results * These can be suOTnarized as : 

(1) The inherent advantages of empirical evidence to Identify and 
justify program objectives as opposed to primary dependence on rationally^ 
or logically 5 derived lists of goal statements* The latter can represent 
no more than a first step in identifying a domain of possible outcome vari- 
ables to be assessed q^uantitatlvely and not the end— product (i*e*5 perennial 
conclusions of investigatory efforts citing "the pressing need for clear 
definition of objectives” represent no more than ^motherhood” statements 
where they lack understanding ofs and reference to* measurement processes 
required to construct a viable system of goals )« 

(2) The need for reasonable u niformity of measured objectives as 
essential to the Interpretation and understanding of research and evaluative 
conclusions. Selection of obJectiveB or dependent study variables predomin- 
antly on an ^*ad J-ib” basis 5 following from an investigator's hunehes , or 

the ready availability of information ^ can obsetire comparability of 

findings between studies and between similar youth-work training programs . 
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Even where practiical demands force limi "tat ions in *bhe choice of measirred 
ohjectivesj or new meas-ures are sought ^ better estimates can be made of 
the value of eny variables chosen if these stem from some knowledge of the 
goal dimensions into which outcome variables are expected to fit. 

(3) The advantage of continuity fn monitoring program objectives and 
feeding back information. Standards are not immutable I Measured objec- 
tives are known to possess dynamic nualities which resTilt in differenees 
in their applicability , or meaning^ at different points in time 
as immediate 5 intermediate and long--term objectives) and as a result of 
modifications in program goals that follow from the feedback of evaluative 
results , 

ik) The advantages of using multiple variables to encompass goal 
dimensions that define complex behavioral outcomes. Patterns of inter- 
relationships among criterion variables ean be used to define a common 
core of meaningful, groupings from which composite criteria can be constructed 
for objectifying a variety of desired goal statements. 

The findings of the present study, based on bross— sectionally obtained 
data, have indicated that goal dimensions differ in their coherence and 
valves as a f miction c ^ the type of criterion group under study and the vari- 
ables applicable to that group. Thus, a sample of former youth-work training 
program enrollees , who have had employment experience, yield the most readily 
interpretable clusters of outcome variables , T]ie distal or intermediate 
criteria obtainable from such a group clustered on si^ rather clearly identi- 
fiablaj factors. Dominant among these — and potentially the best composite 
criterion scale — -was a ”Job Success and Adjustment” factor. This was defined 
primarily by the presence of various Job performance criteria, along with 
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several adjustment measL-ires; all of which Gharacterized the enrollee who 
was more likely to enter a white-collar organization after leaving the 
training program* A more specific criterion factor was found to describe 
a ^*Job Stability-Mobility" clusters with the interpretation 
indicating that the more mobile in Jobs and residence 5 the less Job- 
proficient the former enrollee tended to be, "Striving for Vocational 
Success" is another Job-oriented dimension for which the variables describe 
performance outcomes for an ex— enrollee employed in a better quality Job, 
who has performed well and desires to be upwardly mobile; with the last 
of these relatively independent \ Dcational-perf ormance Glusters foimd 
to define degree of Job success for the former enrollee who tended to 
be employed in a blue-collaj? industrial setting and who had a higher quality 
job as well as having obtained more raises. More distinctly adjustment- 
oriented factors were also uncovered. One dealt with aspects of "Community— 
Family Adjustment" and was descriptive of an ex-enrollee who has fewer 
problems with police ^ family and other community members. The other was 
definable as a "Vocational Adjustment" pattern for enrollees who had 
changed Jobs. The factor coupled job proficiency and Job capability 
with feelings of vocational dissatisfaction. 

In contrast to the factors found for the former enrollee group, with 
full-time Job experience, the least interpretable and weakest set of usable 
goal dimensions were found for those enrollees who were never employed full 
time after leaving the training program# The four dimensions Identified 
as potentially applicable were "Conmiunity Adjustment," "Realistic Aspiration 
(Confidence) "Striving for Personal Improvement" and "Job Motivation- 
Planning." It was pointed out that criteria usable for evaluating Buccess 

95 
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in this group wouldg logically, be more limited in scope, since a group 
that has never been employed is not readily measToreable on as broad a 
range of performance criteria as those \fho have held employments nor on 
measures that are as "objective” end verifiable • Without development 
of a wider assortment of outaome measures (e^g^., evaluations by family, 
peers , or other coimnunity membere ), doubts have been raised about the 
applicability of meaningful standards of "success” for a group that has 
failed to obtain employment over a perioi of monthSj, or years^ after leaving 
the youth-work training program. 

The shorter-term or immediate outcome measures available after the 
enrollee has been in the program for some months (i.e., "completed” the 
pro^am) , were found to be adequate for further developmental application 
Tinder five reasonably coherent dimensions. These were defined as "Tx^ainii z 
Program Adjustment” (representing the dominant one, and best potential 
composite scale), "Social Adjustment,” "Job Aspiration Level,” "Planning 
ComT^etency" and "Personal Image.” Here again, however, there are logical 
constraints foimd in the range of variables applicable to a definition of 
enrollee "success” at the completion of a youth-work training program. The 
majority of the outcome variables were confined to subjeetive, attitudinal 
measures dependent upon responses provided by the enrollee . Heverthelees , 
some reasonable set of "externally” derived outcomes were also shown to 
be of value, in the form of e valuations by program professionals ( ccunselors ) , 
work supervisory personnel and peers (fellow enrolleea); in addition to 
information from program administrative records (work site absences) and 
law enforcement agencies (police contacts). (Certainly, where a given program 
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includes specific skiix components , dealing with remedial skills for 
example 5 it is assumed that availahles formal^ verbal and arithmetic tests 



would be incorporated in any mix of short--term criteria . ) 

Recommendations for future research efforts are concerned with three 
forms of additional analysis considered essential to more complete devel- 
opment and understanding of the criteria Identified. These are: 

(a) Explicit determination of the equivalence of the shorter— term 
criteria^ on the basis of their correlations with the longer-term outcome 
measures . This important form of analysis requires the more time consuming 
longitudinal study desi^, with follow-up of the same enrollees over some 
acceptable time period beyond program "eompletion , " The step is critical, 
^ih.ce the extent to which the more quickly and readily obtainable end—of— 



pro^am criteria are of value, in an evaluative system, is wholly a function 
of" their relationship to longer-term (higher priority) objectives i 

(b) Evaluation of the role of the criterion measm*es (or their dimensions) 
S' predl ct or ^cr it er ion system as a means of enhancing the vinder standing 



of criL^rion value arid meaning. It is ciorrantly expected '^hat this would 
be accomplished by utilizing a battery of guidanea tests 'inder development 
for disadvantaged adolescents (Freeberg, 19T0) which, adong with other 
measiares , would constitute the predictors for such a system. In effect 
this not only provides an opportimity to validate the measures of a newly 



developed test battery but at the same time aids in clearer definition of 



the most effective (i.e., predictable) program outcomes. 

(c) A continuing search for criterion biases > as a consequence of sub- 
group differences, that mi^t affect the *'true*’ criterion scores achieved. 
Analyses to determine various forms of ^psychometric bias" (Rock, 1970 ) 
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would req.uire adeq^uate sample size in order to demonstrate the possible 
systematic biasing of* score distributions^ over a variety of* subgroups of 
interest 5 beyond the limited male--female comparisons briefly pointed out 
in the present study. 

On the basis of experiences in data, collection for the present study, 
an additional class of recommendations g bearing on administrative concerns 
seem pertinent. Ihese touch on one of the most crucial problem areas 
in the application of objectives for youth-work training program research 
and evaluation — x-e., the availability of outcome information. As indicated 
previously in this paper, data gathering problems can be xinusually difficult 
to overcome when the samples are composed of poverty -level adolescents , 
many of whom are likely to be minority group members living in ucrban ghetto 
areas . Hiere are also ethical considerations that have been discussed which 
can further limit data availability and add to sample biases , or nonrepresent- 
ativeness . 

It is worth stressing again that no research or evaluation effort can 
be any be tter th^i the guilty of t he criterion data available ^— and further- 
more that the quality of those data can be compromised seriously when tliere 
are severe limitations in the ability to obtain needed follow-up Information. 
Data availability for former enrolls e populations of youth -work training 
programs is seen as a problem to b e ameliorated largely by administrative— 
progreOTmatic means g rather than passively acceptedg or compensated for^ 
solely by research design and statistical method. In this regard g the follow- 
ing general suggestions are made in order to enhance data gathering for a 
system of outcome measures. (Where feasible ^ there may be an opportunity to 
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implement several of these reconmiendations in a presently anticipated 
longitudinal 5 follow-up study effort dealing with test validation and 
further criterion development.) 

The major need is for the establishment of a national staff of 
prof ess iona3.1y trained interviewers (l.e^, data gathering "specialists”)^ 
located in the specific geo^aphlc areas where data pertinent to youth- 
work training programs are most likely to be colleGted. ^Hiese individuals 
should be responsible for ongoing g periodic data gathering from present 
and former trainrng program enrollees . The natm^e of the data req^uired 
for a criterion system g the sampling procedures and the points in time when 
the data are to be obtained should be specif led with reasonable wiiformity. 

A second suggested requirements essential to the implementation of 
the first g is dependent upon the abilit 3 ?' of data gathering personnel to 
maintain periodic contact with former enrollees . Attempts to maintain 
eonbaet , although difficult to Implement and not a complete solution to 
sampling problems can^ in the long run^ be far less costly and more beneficial 
tb program needs than the conventional nonsystematic ) mear'^ required 

to locate samples of former enrollees for each new research and evaiuation 
purpose.. Among the recommended techniques that might be attempted are: 

First, an explicit expression of intent, comaunieated to each enrollee by 
work training program personnel, that efforts will be made to maintain contact 
with him in order to find out how he is "getting along" vocationally and to 
solicit his opinions about the value of the program* He could ^ for example^ 
be apprised of the possibility of his being contacted (e.g,, six months or 
one year) after he leaves the program in order to obtain such Information 
and for which he would be paid if he chose to participate. Second g and in close 

O 




conjxinctlon with this intent would he the n ee ess It to offer a reason 
for maintaining such contact^ with some defined ”pay off” for the enrollee , 
other than finanGial payments alone. Possible ways of accomplishing this 
might be by providing forms of desirable post-program services. These 
coulds for example, range from a brief newsletter (eontaining vocationally 
useful information and aecompanled by a return post card on which categories 
of other desired information can be checked) , to notification of in-person 
vocational guidance or Job placement services that are readily available 
to the former enrollee through the youth-work training program or the State 
Employment Service- The success of any such attempts to maintain contact 
over some reasonable post— program period i^ contingent largely on the 
regularity and quality of the services that can be Instituted* 

Ultimately, the ability to select objectives, tailor them to changing 
program goals and apply them meaningfully for evaluative purpoBes , must 
arise from reasonable access to forms of information essential to the 
measurement of those objectives* Planned data collection procedures, that 
provide continuity in the flow of Quantitative information, serve as the 
cornerstone for establishing and conducting defensible youth-work training 
programs • 
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Criterion Variables 



1 . 

2 . 

3. 

4. 

5 . 

6 . 

7 ^ 

8 . 

9 ^ 

10 . 



11 , 

12 . 

13- 

l4. 

15- 

16 . 

17. 

18. 

19. 

20 , 
21 , 
22 . 

23 . 

24 . 

25- 



26. 

27- 

28 . 

29- 

30. 

31- 

32. 



(Program Completion) 



Variable 



Questionnaire Item No . 



Vocational Adequacy: Total Scale, 

Quality of First Job Desired 

Reason for Selecting Job ( Irrelevant /Relevant ) , , 

Auareness of Job Charaeteri sties Scale 

(Total Score) 

Knou-ledge of Job Selected 

Ability to Do Job 

Starting Salary Expected 

Highest Salary Expected , 

Ways to Look for. Job (Ho. of Relevant Responses) . 

Important Things to Ask Job Interviewer 

(Response Adequacy) 

Job Search Plans (Suitability) 

Long Range Plans (Job Quality) 

Ability to Accomplish Plans (Relevance of Response 

Get Along with Family 

Family Feelings about Enrollee . 

Financial Assistance to Family 

Save Money , . , , , 

Number of People Giving Enrollee a Hard Time . . . 

Health Problems (No/Yes) 

Amount of Trouble with the Police 

Importance of Keeping Out of Trouble 

Niuriber of Jobs Chosen for Training . 

Willingness to TaJee Training Full Time 

Willingness to Train Part Time 

Job Motivation Scale (Total Score) 



. 10 through l6 

... 17 

. . . 18 

, l8(a) through l8(g) 

... 19 

, , , 20 
... 21 
. . . 21(a) 

... 22 
... 23 

... 24 

... 25 

... 26 
, . , 32 

... 33 

... 34 

, - , 35 

... 36 

, . . 37 

... 38 

, . , 4l 

Job Lists (page 10) 

, , , 44 

, , , 45 

46(a) through 46(g) 



Ext er n al In for mat i on 



Data Source 



Counselor Rating Scale. 11 item scale :Appendix 

Work Supervisor Seale 10 item scale : Appendix 

Actual Police Contacts Local agency 

Number Work Site Assi^ments NYC Project 

Number Work Site Absences NYC I^oject 

Peer Rating Score Peer Rating Scales :Appendix I 

Number of Times Chosen for Peer Rating 
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Criterion Variables 



(Post-Progrem: No Employment Experience) 



Variable 

1, Number of Interviewer Contacts Req^uired 

2, Number of Places Lived Since Left NYC . , 

3, Level of Current Activity, 

4, Level of Short-Term Plans . - . 

5, Part Time Work (No/Yes). 

6, Job Quality Level Sought , 

7, Reason Picked Job Sought (Irrelevant/Relevant) 

8, Knowledge of Job Sought * 

9, Ability to Reform Job Sought 

10, Reason for Ability (Irrelevant/Relevant) * 

11, Starting Salary Expected * , , . 

12, Highest Salary Expected 

13, Ways to Look for Job (Degree of Relevance) 

14, Things to Ask About Job 

15, Concern for Job Characteristics (Total Seal 

16, Long Range Work Plans (Quality), , , , , , 

17, Means to Achieve Plans 

18, Get Along with Family , 

19, Future Financial Contribution to Family 

20, Number of People Giving Hard Time 

21, Health Problems (No /Yes) 

22, Number of Times Visited a Doctor 

23, Amount of Trouble with Police 

24, Credit Buying (No/Yea) - , . 

25 , Ease of Obtaining Credit 

26, Visits to State Employment Service , , , - 



Questionnaire item No . 

No Item Upper right 
on quest . cover page 



cor 



e) 



11 



25 (a 



10 

(Page 9) 

15 

16 

17 

18 

19 

20 

20(a) 

21 

22 

23 

24 

through 25(g) 
26 

27 

28 

29 

30 

31 

32 

33 

37 

38 

39 



External Information 
27. Actual Police Contacts 



Data Source 



Local agency 
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Criterion Variables 



1 . 



2 . 

3. 

4. 

5. 

6 . 

7. 

8 . 

9. 

10 . 
11 . 
12 . 
13. 
14., 
15 • 
16. 

17 . 

18. 

19, 

20 . 
21 , 
22 . 

23. 

24. 

25. 

26. 

27. 

28. 

29. 

30. 

31. 

32. 

33. 

34. 



35. 

36. 



(Post-Progr^: Employment Eicperience ) 



Variable 



Que s tl onn ai r e Item No . 



mnnber of Interviewer Contacts Required 

of Places Lived. Since Left HYC . . > • . 

Employed Now (No /Yes) 

Industrial Category (White Collar/Blue Collar) 

Job Quality (Present or Last Job) 

Hours Worked/Week 

Length on Present (or last) Job 

Number of Jobs Held Since NYC ........ 

Reason Left Job (Poor/Good) .... 

Time to Find First Job 

Number of Sources Used for First Job , . , - 

Number of Places Interviewed 

Number of Applications Filed 

Met Job Expectations 

Starting Salary • ^ 

Number of Salary Raises • 

Amount of First Raise * 

Promotion (No /Yes) 

Job Satisfaction Scale (Total Score) 

Level of Short Term Plans 

Job Level Sought (Short Term) • • . . . ^ . 

Level of Long-Term Plans 

Getting Along with Family ’ * 

Family Feelings about Enrollee 

Financial Assistance to Fatnily 

Number of People Giving Hard Time 

Health Problems (No/Yes) 

Number Times Visited Doctor * 

Save Money ^ 

Amount of Trouble with Police 

Credit Buying (No/Yes) 

Ease of Obtaining Credit* • > • • .... . . 

Number of Visits to USES .......... 

Permission for Employer Rating (No/Yes) - . . 



Wo Item Crupper right 




quest . cover na^e 

10 

11 

^ . 13 

.14 

16 

■ ’ *. ! 17(a) 

. . . . 17(d) 

18 

. .... 19 

20 

] . . . . 21 

22 

! ! ; ! ! 23 

24 

24 

25 

26 through 34 

36 [36(a)] 

37 [37(a)] 

. .... 41 
43 

44 

45 

46 

47 

. • * . . ~r i 

48 

• . . . . 49 

..... 50 

. * . ■ . 54 

; . - . . 55 

56 

..... 58 



External Information 



Data Source 



Employer Rating 

Police Contacts (No/Yes) , , 



Employer (present 
last J oh ) 

Local agency 
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HYC Bnrollee Questionnaire 
(Prograjifi Completion Group) 



Educational Testing Service 
Princeton, N. J. 



Instructions 



I’m helping the Educational Testing Service of Princeton^ New Jerss 3 ?' do a 
survey of Neighborhood Xouth Corps ^ We would like to find out how people who have 
been in NYC for a while have been getting by since they came into the program. Like — 
how things have been working out for you in general; what you thir^ you've gotten out 
of being in NYC; how you feel about jobs; and what kind of things you would like to 
do from now on. We plan to use what we find out to m^e training prograns like NYC 
better for enrollees □ 

The answers you give to any questions are all confidential and secret and we 
would never give out any information about anyone’s answers « So there is no way 
it would be used to affect you personally. Anything we find out from these inter“ 
views would only be reported for a whole group of people at a time (like a few 
h'^cLred) and we would never use anybody’s name. 

This should take only a little over a half hoiu^ and I’ll pay you $3o00 for 
your time 3 





NIG ENROLIEE QUESTIOMAIRE 
(Program Completion Group) 

Educational Testing Service 
Princeton, N„ J= 



do 

s9 


(1) 




(2) 


V) 


(3) 




(4) 




(5) 




(6) 




(7) 



,to 



(Last) 



(First) 



(Middle) 

(months) 



pM P P 

Pw Psp. 



(City) 



0 



Married 



I I Divorced or separated Widov^^d 

(8) Living with: /single response or appropriate c ^b lnation/ 



n Mother 
I I Father 
□ Guardian 



Children 
(9) Address /where presently 



□ wifs or Husband 

I [ Brother(e) or Sister (s) 
[~| Relatives and/or Friends 

□ Live alone 



I I Stepmother 
[/J Stepf ainer 



Street 



State 



I would like to know how you feel about being able to inake it^ — ^as far as 
jobs go~after you leave NYC o Like- - 

— do you think 

^■^^(lO) You’re going to be able to get the kind of Job you want when you leave NYC 



□ For sure. 
No problem 



Might have some 
chance 



r I Not much chance 



ERIC 



1 j t 2 



■uUi.AiiiK.. 



— do you think 

( 11 ) If you get a decent job you’ll be able to do the work well enough to satisfy 
any boss 



Q Wot much chance 
on most jobs 



( I Maybe on most 



j [ Yes ? On any good 
Job you get 



™do you think 



( 12 ) You -11 be able to get by on a job without a lot of help from the boss 
or the people you work with 



[~l Sure to get by 



on your ovm 
without help 



You may need a 
little help 
sometimes 



You're going to 
need a lot of help 
from other people 



—do you think 



(13) li you get a job where you have to learn something new^ you could le 
enough to do the Job right 



arn 



I ) An3rtimej for 
any job 



I [ Maybe : could 

learn for some 



i j You would have a 
tough time if you 
have to learn 
something new 



-do you think 



(14) Once you get a jobj you*ve got what it takes to get promoted and move ap 
to better jobs 



I I Would be rough 
for you to get 



Maybe on some 
jobs 



p You could get prO' 
moted for sure on 
any 



promotxons 

^-do you think 

(15) You know enough about how to look for Jobs to be able to get one after NYC 

Q No problem o You Q You know something [j Not too mucho You 

know all the im- about looking but need to toiow a ict 

portant ways to would need some more about how to 

look for a job help look for a Job 

— do you think 

(16) Being in NYC has made you feel more like you're going to be able to make it 
in a Job later on 



□ It was no help 
at all 



n It 



was some help 



f~l Yes, It’s helped 
a lot 



ERIC 






0 



(17) After you leave NYC what’s your first move as far as a job goes --What kind 
of job would you_look for /if going into military= -^^What possible job 
after mi lit ary? ^ 

Don't know _ 

^Obtain job choice^ if possible/ 

(18) What ‘8 the most important reason that you picked that job 



/Reason/ 



As far as any Job goes how about things like: 

(I8a) You get a chance to do interesting work 

|~| That’s real ^ May be important: Q Don t really .:are 

important not too sure about that 

(18b) You can get paid what you’re worth on that kind of Job 



[ [ Don’t really care Q May be djnportant: 

about that not too sure 

(l8c) You wouldn't have to v/orry about being fired 



! 1 That ' s real 

import ant 



□ Don’t really care 
about that 



May be important: 
not too sure 



[ I That’s real 
important 



(l8d) You feel like you’re doing work that's worth doing 



That ' G real 
important 



Q May be ^portant: 
not too sure 



p 



Don’t really "sre 
about that 



(l8e) You can get raises make more money fast in that job 



That's real 
important 



May be important: 
not too sure 



(l8f) You have a chance for steady work in that kind of job 



__ Don't really care 
about that 



Don't really care May be important: 

about that not too sure 



□ That ‘5 real 
important 



(iSg) You have the feeling that you're not pushed for mors work than you can do 



O 

ERIC 



That's real 
Important 



n May be Important; 
not too sure 



I [ Don't really care 
about tnat 



(19) How much do you know about what it- takes to do the job of a 



/as given in item #12./ 



j~j Know a lot about Q Know a few things 

that kind of job about what the 

job takes 

( 20 ) Do you think you would be able to do that work right now 

\~ I les □ Not sure 

/If yps7 

(20a) Why do you think you could do that (What reasons?) 

/List reasons/ 



Don't really know 
much about it 



No 



( 21 ) What salary do you think you would make when j--ou start the job 
of a 



/as given in item #12/ 






,r ; 



(21a) How high do you think you could go in pay on that job 

(22) What are the ways you would look for that Job (What would you check out?) 

as many items as given/ 



/hr , 



(23) When you go to apply for the job of 



what would 



/as given in item §Y7j' _ 

be the mos t •unportant thing you woiild ask the interviewer about the job /No 
I ( Don’t know 



Most Important 



ERIC 
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Nesct most important 



(24) If the first job you get doesn't work out and you decide to drop it v/hat 
would you do 



n Just don’t know 



/Try for at least one respons^/ 



[Tt ”look for another or ^-go for schooling** of any sort/ 

(21a) What Job (what coarse in school) 



( 25 ) What kind of work do you plan to be doing ov^er a longer time= = say 5 to I-C 
years from now 



Don't know (don^t plan that far ahead) 
/Try for choice of work/ 



) How wD.ll you get to do that /No prompts: List responses as given/ 



n D.: 



t Icnov/ 



^ look at this list of other guys (girls) enrolled here in NYC with you. 
Which one of them do you know best (even if don't know any too well). 



(Name of Enrollee) 

I Weint to as'^ you some questions about him (her) , Please remember ar^ 
answers you give are strictly private . Nobody in NYC would ever see these 
answers , 



(27) 


How well do 


you know him (her) 






□ Real 


well Pretty good 


I [ Not too well 


(28) 


How lon^ do 


you know him (her) 

(weeks or months) 




(29) 


In what way 


are you friendly with him (her) 





On the job at Outside of the f ^ Both on the job 

NYC only nY’G job only and outside of NYC 



/if known "outside of NYC Job Only," go to item # (3 la// 




lie 



From what you know of this gay (girl) would you say he (she) 

(30a) Can get a job done when the supervisor gives him (her) something to do. 





PT That's just the 
way he (she) is 


Q Some t ime s like 
that 


! 1 That ■ s not the 
way he (she) is 


(30b) 


Gets 


along with other 


people on the Job 






r: 


J That s just the 
way he (she) is 


pi Some times like 
that 


r~| That's n/'', the 
way he (the) is 


(30c) 


Look 


5 for ways to get 


out of doing work 






c 


J That's Just the 
way he (she) is 


PI S omet ime s like 
that 


1 [ That ’ s not the 
way he (she) it 


(30d) 


Stays out of trouble on the job 








That ' s Just the 
vmy he (she) is 


1 j Sometimes like 
that 


That ■ s not the 
way he (she) is 


(30e) 


Gets 


along with the work supervisor 






L 


J That s Just the 
way he (she) is 


pi Sometimes like 
that 


PI] That ' s not the 
way he (she) is 


(30f) 


The kind of guy (girl) 


who could make good on almost 


any sort of job 




C 


j Yes, for sure 


1 j Maybe 


Probabl; not 






/Know ’’outside of NYC Job only^ 




(31a) 


Gets 


along with people 


pretty well 






f 1 That ' s just the 
way he (she) is 


1 1 Sometimes like 
that 


( J That ' s not the 
way he (she) Is 


(31b) 


Wants 


to get somewhere 


and make good 






□ 


That s just the 
way he (she) is 


j j Sometimes like 
that 


i 1 That ' s not the 
way he (she) is 


(31c) 


Keeps 


cjut of trouble 








□ 


That ' s Just the 
way he (she) is 


j~~l Sometijnes like 
that 


' That ' S not the 

way he (she) is 


bid) 


Should make out pretty well on his (her) own after leaving NYC 


1 


P 


Yes, for sure 


f~/ Maybe 


Probably not 



Now look at this list of NYC enrollees again and pick out the person 
you know N?.X T BEST . 



(Name of Enrollee) 



I want to ask you some questions about him (her) . Please remember any 
answers you give are s trictly private Nobody in NYC would ever see these 
answers ,, 

(27a) How well do you know him (her) 

□ Real well P Pretty good 

(28a) How long do you know him (her)_ 



Not too well 



(weeks or mos., ) 
(29s) In what way are you friendly with him (her) 



On the job at 
NYC only 



Outside of the 
NYC job only 



[ I Both on the Job 
and outside of NYC 



/if known "outside of NYC Job Onlyj" go to item, #(31e}/ 



From what you know of this guy (girl) would you say he (she) 

(30g) Can get a job done when the supervisor gives him (her) something to do 



) [ That's just the Q Sometimes like Q That’s not the 

way he (she) is that way he (she) is 

(30h) Gets along wi-th other people on the Job 

[3 That's just the Q Sometimes like ^ That's not the 

way he (she) Is that way he (she) is 

(30i) Looks for ways to get out of doing work 

p[ That's just the Q Sometimes like Q That's not the 

way he (she) is that way he (she) is 

(30.i) Stays out of trouble on the job 

PI That's just the Q Sometimes like Q That's not the 

way he (she) is that way he (she) Is 

(30k) Gets along with the work supervisor 

Pf That ' s just the Sometimes like PI That ' s not the 

way he (she) is that way he (sue) is 




(301) The kind of guy (girl) who could make good on almost any sort of job 





U YeSj for sure 


( [ Maybe 


j 1 Probably not 




/Know ^^outside of NYC Job on.ly”7 




(31e) 


Gets along with people pretty 


well 






1 [ That 5 Just the 


j J Sometimes like 


5 1 That's not the 




way he (she) ' 3 


that 


way he (she) is 


(31f) 


Wants to get somewhere and make good 






1 1 That ' s just the 


P/[ Sometimes like 


1 1 That ‘ s not the 




way he (she) is 


that 


way he (she) is 


(31g) 


Keeps out of trouble 








|/~[ That ' s Just the 


[ i Sometimes like 


j/] That's not the 




way he (she) is 


that 


way he (she) is 


(31h) 


Should make wUt pretty well on 


his (her) own after leaving NYC 




j T YeSj for sure 


Maybe 


j [ Probably not 


(32) 


How have you been getting along 
(parents^ guardian(s)^ spouse) 


with your family since 


being in NYC 




1 1 No Family /Go to item 


#3j7 






Bad- just don't got 


|~~1 Fair- you get 


1 I Get along great v.dth 




along with your 


by with them 


your family - no pro 




f amlly at all 




blems 


(33) 


How does your family feel about 


what you ' re doing now ( 


in NYC) 




They think you're 


1 j They think you re 


They don t think 




doing great 


getting by okay 


you ‘ re doing any- 



thing worth much 



(34) About how much of what you make at NIC do you give to your family /wk» 



(35) Do you save any money from your NYC pay 




I [ All the time 
(out of just 
about every 
week's pay) 



a A lit tie --off end 

on 




Q Don't 
of it 



save any 



( 36 ) On this list that I show you let me know which of these people or places have 
been giving you a hard time lately 



Supervisor at work 

People you work with 

Social worker (Welfare) 

State employment 

NYC couu^elor 

The police or the 
courus 

Neighbors j people you 
knov; on your block 



Lawyers 

Credit collection outfits 

Storekeepers 

____ Sombody in your family 

A hospital, or people that work 

in a hospital clinic (like the 
doctors] the clerks) 

Any others? 



( 37 ) Do you have any ills or problems with your health that bother you 

Q No Q Yes 

(38) Have you gotten iii any trouble with the police since you've been in NYC 
(arrested and charged, or booked) 



No 

/Go to Item #41/ 



I } Just once couple of time^ 



I I More than 
of tiraes 



couple 



( 39 ) An convictions 



i~| More 1 bar. a 
couple 

( 40 ) Have you been put on probation since you enrolled in NYC 



I I None Q One conviction A couple of 

convl ctions 



□ No 



r~| Yes 



(4i) How important is it to you to 



out of trouble with the cops and the law 



I I Real Important I 
you go out of your 
way to avoid troubla 



You usually try 
to steer clear 
unless youi'e 
pushed real hard 



j { Not too important: 
if you get in 
trouble you don t. 
care too much 




Here - a li=t of names of a lot of different jobs people might want to try for 

after i.hey'leaee NTC , It alee telle a }“tle aoout^what^^^ t°ainln/in 2elyof leave 

Usfwitr^oa; (K^greSgS^b-hSTwith reeponbent Mark 
all items chosen ) , , ^ , 

.T^>, T-i-rT. 



MiUES 



1. ' H^ay^Appl iance Repaiman _- Fixes 

things Like washing machines , 
refrigerators and air conditioners. 

2. Lat he Q pera tur - Rons a machine that 
makes metal parts « 

3 . Store Sal e sma n - Sells things to 
customers and makes out sales recoi ds . 

it Printer - Runs a printing press to 
turn out books and pamphlets . 

5 , Co nstr u ction Carpente r - Builds house 
frames'and puts together otter wooden 
parts of buildings „ 

6, Arc Welder - Welds metal parts to- 
gether with electric welding torch, 

7 ■ Personne l In terviewer - Asks q^ues- 
ttons and takes downi information 
from people applying for Jobs, 

8. Booldceeper - Keeps records of money 
and finances for a business, 

MAIES, 

1. Bank Clerk - Handles money that 
people put in and take out of 
the bank,. 

2. Laboratory Technician - Takes care 
of laboratory equipment and does 
tests on chem_lcal samples , 

3. Clothing Store Saleswoman - Sells 
clothes to customers and makes out 
sales records,, 

A. Rpc retary Typist - Types letters 

in an office and keeps things propeilj' 
filed a 

5 a H a i r Stylist - Cuts and sets women's 
hair in a beauty parlor, 

6, Window Dresser - Sets up merchandise 
in store windows to attract customers, 

7„ Personnel Interviewer Asks questions 
and takes down Infomation from people 
applying for Jobs, 



O 




9„ Auto Mecha nic Checks out car 
troubles and fixes them. 

10, Construct io n Steel W orker -^v/orks 
on steel frames of new i.. uildings . 

11, Watch Re pair man - Fixes watches 
or clocks n 

12, D raft smar. - Makes drawings used 
as blueprints for buildings,, 

1^ Trac to r Trailer Driver - drives 
trucks over long distances,, 

_1A. Plumber - Installs pipes and 

faucets in buildings and fixes 
them , 

15, Electr onic T echn ician - Helps 

check out and" put together parts 
of electrical equipment. 



8. Bookkeepe r - Keeps records of 
money and finances for a busi" 
ness c 

9, Teacher's Aid - l.eLps teacher 
teach children in a classrocm, 

10, Practical Nurs e ■ Takes care 
of sick peopl'.- in their homei- » 

n, n>»nt. 1 Pt ‘ s Aegis t ant ■ Helps : 

dentist take care of patients, ; 

12.. Travel Agen t - Helps people make ; 
plans for taking vacation trips, , 

13 „ Stock Clerk - Keeps records of j 
stock and guppUes for a company, 

lA, Telephone Operator - Help people 
make telephone calls, i 

15, Compute r Keypunch 0peLi^O£ " i 

Operate machine to put informa- i 
tion on computer cards. 




( 42 ) Which one of those that you marked would you like best 

(name) 

(43) Which one on the list wouldn't you like at a 1 1 

(name) 

( 44 ) How much would you have to be paid to take training full -tijne for that job 
you liked best? 

I j you wouldn't take training fuU-time for any pay 

r~i Take about the same as NYC pay = /NYC pay /hr .. / 

f ( You would want more than NYC pay„ How much /hr 

(45) Suppose while you were here at NYC you had a chance to take special training 

as a a What would you went to take some 

(Given in item # 42 ) 

training for a couple of hours a nlght--ln your spare time .> 

f~j You would do it for no extra pay 
j j You wouldn't bother 

{ I You would want the same pay per hour as NYC 

1 ^ 

LJ You would want morethm your NYC pay per hour« How much /hr . 

(46) If you could get this Jol. you want as 

(Given in item ^^42) 

when you leave NYC^ what-s the le ast pav you would take^ /hr a 

(46a) How much would you wsait if you had to move out of the city to get that job 
(away from your family and friends) 

Q Wouldn-t matteri you would still take the same pay 

Q You wouldn-t take that job for any money if that is the it was 

Q Take it for more pay: How much more pay /hr ^ 

(46b) Howr much would you w^t If the guy you would work for in that Job was going 
to be on yom back a lot 

Q Woul<toi*t matter; you would still take the aeone pay 

p] You wouldn-t take that job for any money if that^s the way it was 

Q Take it for more pay a How much more pay /hr o 



( 46 c) How much would you want if you had to do some extra work that v;as dirty 
as part of that .job 

I [ Wouldn't matter,; you would still take the same pay 
I 1 Wouldn't take that Job for any money if that's the way it was 

| ~~1 Take It for more pay= How much more pay /hr . 

(46d) How much would you want if you had to. go to schuol at night on your own time 
for special training in that job 

I I Wouldn't matter; you would still take the same pay 

r~| You wouldn't take it for any money if that's the way it was 

f ~|' Take it for more pay^ Hovj much more pay /hr 

(46e) How much would you wanb if you didn't like the people you had to work with 
on that job 

I I Wouldn't matter; you would still take the same pay 

r~t You v/ouldn't take it for any money If that's the way it was 

I [ Take it for more pay. How much more pay /hr c. 

(46f) If you woiild get a fair raise each year but no chance to get promoteu for 
a long time (like 3 years or more) 

□ Wouldn't matter; you would still take the same pay 

1 I You wouldn^t take* it for Bjny money if that*s the way it was 

) [ Take it for more pay« How much more pay /hr . 

(46g) If there was alw^s pressure .on the nob to get a lot of work done 

j [ Wo'uldn't matter; you would still take the same pay 

r°l You wouldn^t take it for any money if that*s the way it was 

^ Take it for more pajo How much more pay /hr. 

Additional coimants 




Appendix C 



Post’=Prograin Questionnaire 

Section I: Former Enrollees with 

Employment Experience 
Section 111 Former Enrollees with No 
Employment Exper i enc e 








Former NYC Enrollee Qtiestionnairc 
(Post^NYC Groups) 

Educational Testing Service 
Princeton, N. J. 

Instructions 

I'm helping the Educational Testing Service of Frincetonj New Jersey do a 
surYey of how people who were in Neighborhood Youth Corps have made out since they 
left the program^ Like — • how things have worked out for you in general; hov much 
vjork you've been able to get; how you feel about jot and what kinds of things you 
would like to do from now on^ We plan to use what we find out to make training 
programs like NYC better for the people who have been in themo 

The answers you give to any questions are all confidential and secret and we 
would never give out any infomation about anyone's answers So there is no way 
it would be used to affect you personally. Anything we find out from these inter^ 
views would only be reported for a whole group of people at a time (like a few 
hundred) and we would neve r use anybody's name» 

This should take only a little over a half hour and I'll pay you S5-00 for 
your t Jjne ^ 



ERIC 
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a 



( 2 ) 

(3) 



(5) 

( 6 ) 

(7) 



( 8 ) 



Former NYC Enrollee_ Queationnaire 
(Post-NYC LTr-oups; 

Educational Testing Service 
Princeton, M. J, 



(l) Name 



(Last) 



(First) 



NYC 



Time since left NYC 
Date of birth ___ 
Place of birth 



(Middle) 



(mos , ) 



] I Divorced or separated ¥idov/ed 

Living with, /single response or appropriate combination/ 



[ [ Mother 
Q Father 
{~~| Guardian 
□ Children 



I [ Wife or Husband 
I [ Brother (s) or Sister(s) 

I [ Relatives and/or Friends 



Phone calls 



Home visits 



Numbei 



Number 

Other contacts 

attempted 

Number 



Pm 

pw 



(Cit ) (State) 

Highest school grade completed (when entered NYC) 

Marital status p g . p Married 



t i Stepnother 
stepfather 



live alone 






(9) Address /where presently llvinjg/ 



( 10 ) 






(Street) (City) 

How many other places have you lived since you left NYC 

(a) How mary?' of these moves within this city _______ 

(b) How many moves outside to other cities ______ 



(State) 



(number; 



■ERJC 



(n) How have you made out since you left NYC 



Q Are you working now /20 hrs . a week or mor^ 

Q Not working now^ hut have held Job since you left NYC /20 hrs . a week or 
more, for at least one wee^ 

I j Not working j have not held Job /Go to section II , Page 9 7 

(12) Where do (did) you work 

/Company nsmie and address- -present or most recent jo^ 

(13) What kind of business is that 

(14) Wiat kind of work do (did) you do 

Title /description where needed 

(15) How many hours a week do (did) you work 

( 16) How long on that Job 

/Weeks or montlm/ 

(17) Was this the first Job since vou got out of MYC Yes D No 



/if not first Job/ 




■ 


(17a) 


How many Jobs held since leaving NYC 






(17b) 


How long were you on each Job 1st 


2nd 3rd 




(17c) 


Job titles for each 








1st 2nd 


3rd 




(I7d) 


Why left each Job 








1st 2nd 


3rd 













(18) Hovf long were you out of NYC before you got the first Job 

/dsys or weeks7 






(19) Which of these did you try to get help from — to get your first job 



f~f NYC project personnel 

Q Told about by fi-iends or 
people in the neighborhood 

□ Saw it in newspaper 
(Want ad) 

[~~1 Eniployment agency wliere 
you pay 

□ Ju st went to companies and 
asked about Jobs 

(l9a) Which one of those was the 



n state employment (YOC) 

[~1 Told about by family 

P~] Saw sign In window 

I I Church and community leaders 
(like store owners, school 
teachers or ministers) 

__ Other 

way you actually got the first Job 



(20) How maiiy places did you actually get to see someone to ask about a job 
before you got the first Job 

(21) In how many places did you actually get to fill out an application before 
you got the first Job 

(22) How close have you come to doing what you thnught you would when you 3.eft 
NYC--as far as your Job goes 

[~1 Much better than you thought you would do 

P[ About what you expected you would do 

P[ Worse than you expected to do 

(23) About how much do (did) you make on your Job per hour when you started 
/for present Job or last Job heldy 

P $1.25 to $1. 50/hr. p $1,50 to $1.75/hr. P $1.75 to $2 ,00/hr. 

P $2.00 to $2. 50/hr. P More than $2,50/hr. 




(24) Did you get a raise in pay on your present (or last) job 



P 



PYes 



Number of raises 



Amount - 1st raise 
2nd raise 



(25) Did you get a promotion on that job 



No 



□ lee 



(26) Wliat are your feelings about the kind of work you do (did) 



P] It's great : Q It * s okay 

].ike it a lot 

How's your pay for the kind of work you do (did) 



a 



Don't like the work 
at all 



□ Good pay for the 
kind of job you 
have (had) 



I I Just about what it |~~j 
should be for the 
Job 



A lot less than 
the Job is (was) 
worth 



(28) If you have your we^ would you want to work for this place five years 
from now 



P] Definitely no 



Do (did) you feel like you're 
belong) 



f~~[ Yes you would 
really part of the company (like you really 



f j Not sure: depends 

on how things go 



I I That ' s Just the 
way you feel 
(felt) 



f I Maybe sometijnes you ^ No— Just another 
feel (felt) that job 

way 



(30) How close does (did) your work come to the way you think a Job should be 



Nowhere near what 
you think a job 
should be 



Somet im es close 
to what a job 
should be 



□ It's just the wsy 
you think working 
on a job should be 



(31) How good is (was) your pay if you compare it with what you could get at 
other companies for the same kind of work 



■4 



n Better pay than 
other companies 
for the kind of 
work 



O 
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l~~[ About the same pay 
as other companies 



r~1 Less than other 
places pay for 
that kind of work 



( 32 ) 



(33) 



(34) 



/For 

(35) 



(36) 



(37) 



O 

ERIC 

Mi’lliiHilfflffTlTLiU 



V^fhen you took a job in the company^ — if you knew then what you know now 
about it --would you have gone to work there 



j [ No — not if you [_J Not too sure [_/| Yes, you would take 

knew about the a job there again 

place what you 
know now 

Based on your age, your ability and your experience, how do (did) you feel 
about your pay 



I j More pay than a 

guy (girl) like 
you would expect 
to make 



□ About where you 

should be in salaiy 



rn Making less than a 
guy (girl) with your 
experience should 
expect to inake 



When you finish a day’s work^ do (did) you feel like you did something 
worthwhile 



( j Almost neYer feel 
(felt) that way 



pn Sometimes true 



j i Alanoat always feel 
(felt) that way 



those presently employed/ 

How high do you think you ^11 be able to gi 

/hr , 

AmoEUit 

/presently^ 
employed^/ 

What plans do you have for 
about the nexl six months or so 
/anyone or combination/ 

Q Keep working on my job 
(No particular plans) 



! 

I 

I 

in p^ on the job you have now i 

I 

1 

i 

! 

/Pr e a ent ly _ | 

unemployed/ 

( 36 a) What plans do you have for 
about the next six months 
or so 

i~T No particular plans 



I I Look for a better job 

I I Try to gat some extra 
training (on the job or 
part--tme school) 

Go to school full time 

What kind of job would you 
look for if you have to leave 
this one 

n Don't know 

for job choice/ 

(Type of job) 




□ 



Try to find pother job 
full-time 



bo school full-time 



n Find a job and go to school 

(37a) What kind of Job are you look- 
ing for, mainly /or what w^ould 
you look for when you do look/ 

|~j Don^t know 
^ry for job choice/ 



-s 

4 



I 




Ji 



(Type of job) 



(38) 'vilhat's the most important reason that you picked that job 



/Single response/ 

(39) How iruch do you know about what it takes to do that job /As given in 37 or 37(al/ 



□ A lot about that 
kind of work 



Know a few things 
about what that Job 
takas 



QJ Don ’ t re ally 
know much abont 

It 



(40) What would you do to get that next Job /Prompt only by asking ’'Anything else’'? 
List as many items be 



) What kind of work do you plan to he doing over a longer time-- like five 
to ten years from now 



Don't know (don't 
plan that far 
ahead) 



j~~[ Same kind of work ^ Choice of occupa- 

you're doing now (or tion 

did in previous Job) 



(Single choice J" 

/if choice given/ 

(41a) How will you get to do that /No prompts: List responses as give^ 



Don't know 



( 42 ) Are you going to school now 
P No 



rn Full-time (20 hrs. Q Part-time (less than 

or more a week) 20 hrs . a week) 



/if in sc 

(42a) What kind of course are you taking 



(Nam© of course or type of training 
program) 



(43) How have you been getting along with your family since leaving NYC ^ (Parents, 
guardlan(3) , spouse,) 



P No family . , , , /Go to Itrai §k§i/ 



ERIC 



P Bad — Just don't get 
along with your 
fasnily at all 



Fair, You get by [P 

with them 



Get along great 
with your fairaiy- 
no problems 






0 



( 44 ) How does your fairdly feel about how you ^ re doing 

Q They think you’re Q They think you’re Q They don’t think 

doing great getting by okay you're doing any- 

thing worth much 

(L 5 ) About how much of every ten dollars you make do you put toward the family income 
/or how much when you did wor^ 



(Amount) 

(46) On this list that I show you let me know which of these people or places 
have baan giving 3 tou a hard time lately. 



_____ Supervisor at work 
_ . People you work with 

Social worker (Welfare) 

State employment 

_ School (IJJce teachers 

or other people that 
run the school) 

The police or the 

courts 

Any others 

(47) Do you have any ills or problems with 



La‘v0'ers 

Credit collection outfits 

Storekeepers 

Somebody in your family 

A hospital^ or people that work 
in a hospital clinic (like the 
doctors] the clerks) 

Neighbors or other people you 
know in your neighborhood 



your health that bother you 



D 



No 



(48) About how many times did you go 
since you've been out of NYC 



to see a doctor—because you were sick — 



Nunber 



(49) Have you saved any money since you left NYC 



□ All the time P Some — off and on | j Just about nothing 

(out of just about saved 

every paycheck) 

(50) Have you gotten in any trouble with the police since leaving NYC 

□ Just once P A coupls of tarns s | ~ j Mors than a coupl© 

of times 



m2 



No 



ERIC 



/Go to item # 52 / 






(51) Any convictions since you left NYC 



Q None One conviction 



A couple of 
convictions 



j~| More than a 
couple 



(52) Have you been on probation since leaving NYC 



No 



a 



Yes 



How important is it to you to keep out of trouble with the cops and the 
law 



{ I Real important: 
you go out of your 
way to avoid trouble 



(54) Do you buy things on credit 



No, Don‘t use it 



P| You usually try 
to steer clear 
unless you* re 
pushed hard 



(~| Not too important .. 
If you get in 
trouble with the 
copSj it doesn^t 
matter much 



[""} Some things* Like Q Most everything 
expensive ones. 



(55) Any trouble getting credit for buying what you want 



Can get it almost 
an 3 rtime for any« 
thing 



You can get credit pj You have a tough 
for a few things if tims getting credit 
you need it for almost anything 



(56) Did you visit the State Employment Service since you got out of NYC^ 



No 



□ Yes . , 



3 er of lasits) 

(57) Did you receive any unemployment payments during the year? 



P 



No 



q 



Yea . 



(Nraiber of weeks) 



(58) Can we send theae questions to the place where you work to aek how^you^re 
doing there? ^Show Employer Rating Scale* STRESS CQNFCTOEMT lALIT Y/ 



t] No Qres 

/if yea/ Supervisor's Neiiie 
Additional nonments by Former Enrollee 



O 
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/Secti on II/ 

/Not Working] Have Not Held 



Are You: 



I I Mostly in school /20 hrs. a week or more/ 

Q In another work-^tralnlng program (like BfflTAj Job 
JOBS) 




jj Still Job hunting 

Not vjorking--not looking 
[ I Housewife 
Q Going into military 
[3| Other 



r. oo/Go to Item 15(al/ 



(12) What school (or work-training program) are you in 



(Name or type of school or of 
work -training program) 



(13) How long after* NYC did you get into that 



(Time after NYC 1 ji Weeks) 
{14) What major course are you taking? (^at job in wo rk^t raining program) 



(15) What short-term 
from now 



plane do you have— -like for the next six months or a year 



[~| Stay in school (or training program) 



Look for a job 



□ Not 



sui'e 



(15a) What ahort-tem 
year from now? 



plans do you have—like for the next elx months or a 



I j Go to school full time 



□ Look (or start looking) for a Job 



ERIC 



[I] Go to school part time □ Not sure (no special plana) 

jL Im- 



□ Yes 



(1 6 ) Have you done any part-time work (off and on) since you left NYC □ No 
/If Yes/ 

(l 6 a) What kind of part-time work 

(Describe type of Job) 

(l 6 b) How much do (did) you get an hour for that /hr , 

(17) Wien you look for your first full-tima job what kind of job woiild you look for 
(Even if remaining in school or if going into military) 

[ ^ 1 Don ^ t know 

/(obtain some Ghoice^ if possibl^ 

(Type of job) 

(18) What^s the most important reason that you pick that 

(Reason) 

(19) How much do you know about what it takas to do the job of a ^ 

(As given in # 17 ) 

Q Know a lot about P Know a few things Q Don^t really know 

that kind of work about what Job much about it 

takes 

(20) Do you think you would be able to do that work right now 



P Yes 

/if Yes/ 




sure 




( 20 a) Why do you think you are able to do that jq^i 



(What reasons) 



( 21 ) 



What salary do you think you would make when you start the Job as 




a 

(As given .in # 17 ) 



(22) How high do you think you could go in that Job in pay? /hr ^ 

(23) What are the ways you would look for that Job? /Lia^ 



Don"t know 







( 24 ) When you go to apply for the job of what would be the 

(As given in #1?) 

most important thing you. would ask the interviewer about it 



Don't know 

/Most Important 



( 25 ) How important to you are these other things to ask about when you go for a job 
interview 

(25a) The pay when you start 

pNot that import^it, Q‘ iC\i xn f ^ Yoxi wou-Xd. s^Xwo^s 

You wouXdn-t neces- some pXaoes ask 

sarily bother asking 

( 25 b) How high you could go if you sbay in that job (po^aibillties for being 
promoted) 



Q Not that important^ Q You might ask in You would always 

You wouldn^t necea-- some places ask 

sarily bother asking 

(25c) How people get along with each other at that company 



pn Not that important .3 
You wouldn^t neces'^ 
sarily bother asking 

(25d) If the job is steady 



You .might ask in 
some places 



Not that important, You might ask in 

You wouldn^t neces- some places 

sarily bother asking 



(25e) How much overt Jjne you have to work 



I 1 Not that important <. You might ask in 

You wouldn^t neces- some places 

sarily bother asking 

(25f) Whathar you get time off to go to school 



Not that important 5 You might ask in 

You wouldn^t nacas- some places 

sarii^ bother asking 





j~| * You would always 
ask 



I~1 You woiild always 
ask 



n would always 
ask 



I f You would always 
ask 



C 25 g) How much vacation and other benefits the company gives--like life 
insurance and hospital zation insurance 

j I Mot that important, You might ask in You would always 

You wouldn't neces- some places ask 

sariJLy bother asking 



(26) What kind of work do you plan to be doing over a longer time - -like 5 to 10 
years from now 

I I Don*t know for choice of oeeupationy' 

(don^t plan that 
far ahead) 

(Choice of occupation) 

(27) How will you get to do that /No prompts: List reponses as given/ 

□ Don * t know 



( 2 S) How have you been getting along mth your family since leaving NYC (that 
is, either parents, guardlan(s), spouse) 



-1 



No family — ^on my own^ ^ 



= /Gq to itCTi 30/ 



P Bad- -Just don’t get Fair— I get by 

along vd-th ny family with them 

at all 



I I Get along great 
with my femiiny— 
no problenis 



(29) When you get a Job, about how much out of flyerv ten dol 1 ara you make would you 
be willing to put toward the fand.ly income 



(30) On this list that I show you let me know which of these people or places 
have been giving you a hard time lately 



Supervisor at work 

People you work with 

Social worker (Welfare) 

State employment 

School (like teachers 
or people that mm the 
school) 

The police or the courts 

Any others 



Lawyers 

Credit collection outfits 

Storekeepers 

Somebody in your faniily 

A hospital, or people that work 
in a hospital clinic (like the 
doctors, the clerks) 

Nei^bors or other people you 
know in your neighborhood 




( 3 1) Do you have any ills or problems with your health that bother you 

P~T No Yes 

( 32 ) About how many times did you go to see a doctor=because you were sick- 

since you' ve been out of NYC 9 

(Number) 



(33) Have you gotten in any trouble with the police since leaving NYC 
(ar'rested and charged or booked) 

D No Q Just once Q A couple of times Q More than a couple 

- ^ of tijnes 

/Go to item^^ 



(34) Any convictions 



{ I None One conviction 

(35) Have you beeri placed on probation ainGe leaving NYC 



□ A couple of P More than a 
c on vi c t i ons c o uple 



Yes Q No 

(36) How important is it to you to keep out of trouble wj.oh the cops and the law 



I [ Real Jjnportant: 
you go out of j^our 
way to avoid trouble 



(37) Do you buy things on credit 
{ [ No^ Don't use it. 



j \ Usually try to 

steer clear unless 
you’re pushed hard 



r~l Some things -like 
e^ens i ve one s 



(33) Any trouble getting credit for buying what you want 



j I Can get it almost 
anytime for anything 



Q Can get credit 
for a few things 
if you need it 



P Not too important. 

If you gat in trouble 
with the cops# it 
doesn't matter much 



D Most everything 



|~~~j Have a tough time 
getting credit for 
alinost anything 



( 39 ) Did you visit the State Employment Service since you got out of NYC 



P No 



I I Yes e « e 9 9 » 



A dditional comments by former enrollee 






(Nuntoer of visits) 
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App ^ndix D 



Guidelines Used for Ratings 
of Job Quality 



The Job Quality scale is intended to define Job choices on the basis of 
a combination of Job status ^ skill levels and potential for advancement . 
The jobs categorized below' are examples of occupations that represent 
each of the three skill levels at which the given job is to be rated. 



Level '^l” ^ Unskilled Jobs : Generally dead end and/or lo-w-level 

employment^ requiring little skill; generally low paid. 

Dishwasher 

Porter 

Nur a e ■ 3 Ai de 

Hospital Attendant 

Laborer 

Car Washer 

Baggage Hanger 

Service Station Attendant 



Level " 2 *^ Sera is killed ; Requiring some degree of verbal or other 

technical skills | moderate pay (or moderate status if 
low skill). 

Miite Collar Clerical 
Truck Driver 
Computer Keypunch 

Sales Person - (store sales > door-to-door) 

L'lmiber j ack 
Shipping Clerk 
Typist 
Waiter 

Practical Hurse (LPN) 



, Level ”3" Skilled Trades ; Semiprofeasional ^ j^ofesslonal 

i Secretary-Stenographer 

I Mirse (RH) 

I Baker^ Carpenterj Mason 

i Plumber^ Eleetrician 

i Auto Mechanic j Itochinist 

I High-Level Sales (e.g.^ insurance^ scientific^ or medical equipment) 

I C^Kputar Operator or Pro^ammer 

I Lab and Medical Technician 



lERlC 



M'iB 



Counselor's and Work Supervisor's 
Rating Scales 





NYC Project 




o 



COUNSELOR'S RATING .SC AIE 



Neighborhood Youth Corps 



,\fl Counselor's Nane Name of &iroHee 

V) —— — ^ ___ ^ Being Rated 

, Kow many months do you 

know this enrcllea 

— 



On this sheet are 11 statements about enrollee behavior that coijiiselors conrider impo 
tant in deteritiining hov; well an enrollee is coming along. We would appreciate your" 
evaluation of how these apply to this enrollee. 



Please read each statement carefully . Then circle one of the numbers 1 to 5 that best 
indicates how the stabemant applies to the enrollee. 



1 2 3 4 5 

This describes This is Somet im es This is The enrollee 

the^eni-ollee trae most this is not is not like 

perfectly of the ti’ue of the usually this at all 

tijne enrcllee true 



Circle one 



1 . 

2 . 



Pays attention to good grooming and dresses appropriately. 

Is not very open about discussing personal and job 
problems. 



3- Shows a lot of resentment and hostility. 



1 

1 

1 



3 

3 






4. 




Is cooperative and willing to listen to advice. 
Makes realistic pl^s about future jobs. 

Shows little poise or sslf—assuranca. 






7. 

. 8 . 



; o 


9. 


i o 




lo 

i 


10, 


[rr- 


n. 



E ■ o 

I eric 



Is coherent in ejqpresslng himself (herself) . 

Is motivated to want to %vork and e^send effort , 

Does not show good day-to-day planning so that he (she) 
can handle the job (let's home life interfere, for 
example) . 

Indicates a willingness to eni*oll in school or some sort 
of training on a part-tjjne basis. 

Generally shows unhappiness with the v;ork site assignment 
and wants to change. 




1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
12 3 4 5 
1 2 3 4 5 



1 2 3 4 5 I 

1 2 3 4 5 



1 





WORK SUPERVISOR'S RATING SCALE 






Neighborhood Youth Corps 




b 


(Male Enrollee Form) 








NYC Project 


\0 Work Supervisor's Name 


Name of 


Enrollee 




You Are 


Rating 




How mexiy months has the 






enrollee worked for you 





Below are 10 statemenis about things that work superYisors consider important when it 
comes to how the erirollee is doing. We would appreciate .your telling us how each one 
applies to this enrollee . The information is strictly pr ivate and will not have any 
effect on the enrollee in any way. 

Please read each statement careful].y ■ Then put a check in one oi the five boxes 'co 
show how that statement fits the enrollee that youdre rating. 



1* KNOWS HOW TO FOLTOW INSTRUCTIONS PROPERLY. 





This describes 


This is true 


Sometimes this 


This is 


not 


The enrollee 




just how the 


most of the 


is true of the 


usually 


so 


is not like 




enrollee is 


time 


enrollee 






this at all 




□ 


a 


□ 


□ 




□ 


2. 


PJESENl’S TAKING ORDERS FROM THOSE MIO SUPERVISE HIM 










This describes 


This is true 


Somatlines this 


This is 


not 


The enrollee 




just how the 


meet of the 


is true of the 


usually 


eo 


is not 1 iKe 




enrollee is 


tiiTie 


enrollee 






this at aJi 




□ 


□ 


□ 


□ 




□ 


i 3. 


KNOWS HOW TO DRESS RIGHT FOR TEE 


JOB. 










This describes 


This is true 


Sometimes this 


This is 


not 


The enrollee 




just how ths 


most of the 


is true of the 


usually 


so 


is not like 


enrollee is 


time 


enrollee 






this at all 


f 

€D 


□ 


□ 


□ 


□ 




□ 














> 

j: o'*’ 


TAKES SOME TOIDE 


IN TIE WORK AND 


DOESN'T JUST RUSH 


THROUGH TO 


GET IT 


FINISHED 
The enrollee 


This describes 


This is true 


Sometimes this 


This is 


not 


i o 

t 


just how the 


most of th# 


is true of the 


usually 


so 


is not like 


enrollee is 


time 


enrollee 






this at all 






□ 










i; 

1 


□ 


P 


□ 




n 


f o 














ERIC 




*:■ . 


■ 









- 2 - 



5. HAS To BE TOLD WHAT TO DO EVERY MINUTE OR m CAN'T KinEP BUSY. 



This deacriDes 
just how the 
enrollee is 


This is true 
most of the 
time 


Somet-iizias this 
is true of the 
enrollee 


This is 
usually 


not 

so 


The enrollee 
is not like 
this at all 


□ 


□ 


□ 


rd 




□ 


6. GETS ALONG WITH OTHERS ON Tffi: JOB. 








This describes 
Just how the 
enrollee is 


This is true 
most of the 
time 


Some times this 
is true of the 
enrollee 


This is 
usually 


not 

so 


Tile enrollee 
is not like 
this at all 


□ 


d 


□ 


□ 




d 


7. CAN'T GET TO WORK 


ON TIME. 










This describes 
just how the 
enrollee is 


This ia true 
most of the 
time 


Sometljnes this 
is trr.e of the 
enrollae 


i’hid Is not 
usi\ally so 


The enrollee 
is not like 
this at all 


□ 


Q 


□ 


□ 




□ 


8. SHOWS SOME INITIATIVE IN TAKING 


ON A PIECE OF WORK. 








This describes 
just how the 
enroUee is 


This is true 
most of the 
time 


Sometimes this 
is true of the 
enrollee 


This is 
usually 


not 

so 


The enrollee 
is not like 
this at all 


□ 


□ 


d 


d 




D 


9. DOESN'T MAKE TROUBLE ON THE JOB, 










This describes 
just how the 
enrollee is 


This is true 
most of the 
time 


Sometimes this 
is time of the 
enrollee 


This is 
usually 


not 

so 


The enrollee 
is not like 
this at all 


□ 


□ 


P 


P 




(P 


10, ASIffi QUESTIONS IF 


PROBm© GOMEL 


UT— DOESN'T JUST GO 


AHEAD AND DO THE 


JOB WRONG, 


This describes 
just how the 
enrollee is 


This is true 
most of the 
time 


Sometljiies this 
is true of the 
enrollee 


This is 
usually 


not 

so 


The enrollee 
is not like 
this at all 


□ 


d 


□ 


d 




P 



O 







0 



WORK SUmRVISOR'S RATING snAT.R 
Neighborhood Youth Corps 
(Female Enrolloa Form) 



NYC Project 



Work Supervisor's Nsme 




_ Name of Enrollea 

You Are Rating __ 

How many months has the 

enrollee worked for you 



Below are 10 statements about things that work supervisors consider important when it 
comes to how the enrollee is doing. We would appreciate your telling us how each one 
applies to this enrollee. The information Is strictly private and will not have any 
effect on the enrollee in Bjay way. 



Please read each statement carefully . Then put a check in one of the five boxes to 
show how that statement fits the enrollee that you* re rating. 



1. SHOWS SOME INITIATIVE IN TAKING ON A PIECE OF WORK. 



This describes 
just how the 
enrollee is 



This is true 
most of the 
tJjne 



Sometimes this 
is true of the 
enrollee 



□ □ □ 



This is not 
usually so 

□ 



The en^^ollee 
is not like 
this at all 

□ 



RESETnITS TAKING 


OmERS FROM THOSE WHO SUFEEVISE HER „ 






This describes 


This is true 


Sometimes this 


This is rot 


The enrollee 


just how the 


moat of the 


is true of the 


usually so 


is not like 


enrollee is 


time 


enrollee 




this at ail 


P 


p 


□ 


□ 


D 



1 




QO 

O 

O 



•• o 

-fEklC 

MMilRlffTIfliU ^ 



SHOWS INTEREST IN lEARNING KOm ABOUT THE JOB 



This describes 
just how the 
enrollee is 



This is true 
most of the 
time 



Sometimes this 
is true of the 
enrollee 



a □ ■ □ 



CAN'T GET TO WORK ON TIME, 



This describes 
just how the 
enrollee is 



This is true 
most of the 
time 



Sometames this 
is true of the 
enrollee 



□ n__ i.45 a 



This is not 
usually so 



The enrollee 
is not like 
this at ail 



□ 



n 



This is not 
usually so 



The enrollee 
is not like 
thxs st ell 



D □ 



- 2 - 



5. HAS TO BE TOLD WHAT TO DO EVERY MINUTE OR SHE CAN'T KEEP BUSY. 





This describes 
just how the 
enrollee is 


This Is true 
most of the 
time 


Sometimes this 
is true of the 
enrollee 


This is 
usuallj?' 


not 

so 


The enrollee 
is not like 
this at all 




□ 


□ 


□ 


n 




Q 


6. 


ASKS QUESTIONS 


IF PROBLEM COME 


UP— D0ESN‘>T JUST GO AHEAD AND 


DO Ttffi 


JOB WRONG. 




This describes 
just how the 
enrollee is 


This Is true 
most of the 
time 


Sometimes this 
is true of the 
enrollee 


This is 
usually 


not 

so 


The enrollee 
is not like 
this at all 




□ 


a 


□ 


□ 




□ 


7. 


IS OFTEN ABSENT FROM WORK . 












This describes 
just how the 
enrollee is 


This is true 
most of the 
time 


Sometimes this 
is true of the 
enrollee 


This is 
usually 


not 

so 


The enrollee 
is not like 
this at all 




a 


□ 


D 


D 




□ 


8. 


SHOWS SOME PRIDE IN THE WORK AND 


DOESN^T JUST RUSH 


THROUGH TO 


GET IT 


FINISHED 




This describes 
just how the 
enrollee is 


This is true 
most of the 
time 


Sometimes this 
is true of the 
enrollee 


This is 
usually 


not 

so 


The enrollee 
is not like 
this at all 




□ 


□ 


p 


P 




p 


9. 


WASTES TME ON 


TIffi JOB. 












This describes 
just how the 
anrollee is 


This is true 
most of the 
time 


Sometimes this 
is true of the 
onroliea 


This is 
usually 


not 

so 


The enrollee 
is not like 
this at all 




□ 




p 






□ 


10. 


CAN BE lEFT ON 


FffiR OWN WITHOUT CLOSE SUPERVISION. 










This describes 
just how the 
enrollee is 


This is true 
most of the 
time 


Sometimes this 
is true of the 
enrollee 


This is 
usually 


not 

so 


The enrollee 
is not like 
this at all 




d 


□ 


□ 


d 




n 






Appendix F 



Employer's Rating Scale 
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Appendix F 

Employar*s Hating Scale 

Educational Testing Servlco of Princeton^ New Jersey is doing a study of young 
adiiLts formerly enrolled in a government sponsored work -training program « We would 
appreciate your giving a brief evaluation of one of your employees (or former 
employees) who was enrolled In on© such program. 

The individual we are interested in is 

We have his (her) permission to ask you for this information « Your responses are 
completely confidential, and will never be identified with you or your organisation- 
nor wiHl they be used to affect this employee any wg2^ 

(l) Do you feei that this employee puts in a decent day''B work 

I I YeSj definitely ( I Generally; not Definitely not 



a? 

fe:. 

|c> 



( 2 ) Does this employee get along w^ith the other workers on the Job 



I I Yes^ definiteJ^ 



Generally^ not 
always 



(3) Would you promote this employee 
□ Yes^ definitely 



j I Maybe j not 
certain 



□ Definitely not 



□ Definitely not 



(4) Woiild yo\i hire this employee again 



U Yes , definitely 



I I Maybe j with a few 
re s er vat ions 



Definitely not 



*• ERIC 
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Appendix G 



Rotated Factor Loadings (varimax) 
Program Completion and Pcst-Program Criteria 



ERIC 



Rotated Factor Loadings (varlmax) 
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Rotated Factor Loadings (Varlmax) 
Post-Program Criterion Yariables 
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